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A  PRACTICAL  DECISION  GUIDE  FOR  INTEGRATING  DIGITAL  APPLICATIONS  AND 
HANDHELD  DEVICES  INTO  ADVANCED  INDIVIDUAL  TRAINING 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  objectives  of  this  research  were  to  (1)  identify  and  evaluate  lessons  learned  from 
selected  Connecting  Soldiers  to  Digital  Applications  (CSDA)  pilot  tests  for  the  development, 
integration,  and  sustainment  of  digital  training  applications  and  handheld  devices  in  Advanced 
Individual  Training  (AIT),  and  (2)  develop  a  practical  decision  guide  for  key  decision  makers 
determining  the  utility  of  digital  applications  and  mobile  technologies  for  AIT  and  developing 
metrics  for  accessing  their  impact  on  training. 

Procedure: 

A  literature  review  was  conducted  to  understand  the  factors  that  need  to  be  considered 
when  integrating  digital  applications  and  handheld  devices  into  training,  as  well  as  identifying 
potential  best  practices  relevant  to  military  training  settings.  To  gain  a  more  practical 
perspective,  subject  matter  experts  were  asked  to  share  their  experiences  with  the  use  of  digital 
applications  and  handheld  devices  in  training.  The  input  from  these  interviews  and  focus  groups 
was  then  analyzed  in  order  to  identify  a  final  list  of  factors  and  subfactors  that  should  be 
considered  as  part  of  the  process  for  deciding  whether  to  and  how  to  integrate  digital  applications 
and  handheld  devices  into  AIT.  These  factors  and  subfactors  framed  the  basic  logic  structure  for 
the  decision  guide. 

Findings: 

Results  from  the  literature  review  and  interviews  and  focus  groups  identified  10  factors 
and  35  associated  subfactors  that  needed  to  be  considered  when  integrating  digital  applications 
and  handheld  devices  into  training.  These  factors  and  subfactors  represented  key  issues  in  the 
areas  of  training  methods  and  delivery,  human  and  contextual  factors,  and  hardware  and 
infrastructure  capabilities  and  constraints. 

Utilization  and  Dissemination  of  Findings: 

The  findings  of  this  research,  in  the  form  of  a  Practical  Decision  Guide,  can  be  used  to 
inform  key  decision  makers  on  the  factors  that  should  be  considered  to  determine  if  digital 
applications  and  handheld  devices  are  appropriate  for  their  AIT  program(s).  A  copy  of  the  final 
guide  was  provided  to  the  US  Army  Combined  Arms  Center  (CAC),  Fort  Leavenworth,  KS,  and 
TRADOC  Deputy  Commanding  General  for  Initial  Military  Training  (DCG-IMT),  Fort  Eustis, 
VA. 
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A  Practical  Decision  Guide  for  Integrating  Digital  Applications  and  Handheld  Devices  into 

Advanced  Individual  Training 


Introduction 

Recognizing  that  more  new  Soldiers  are  entering  the  Anny  with  a  familiarity  and  comfort 
with  interactive  technologies  rarely  seen  in  previous  generations,  the  Army  has  been  exploring 
how  to  exploit  emerging  interactive,  handheld  technologies  and  digital  applications  to  enhance 
the  effectiveness  and  efficiency  of  Soldier  training.  To  this  end,  the  US  Army  Capabilities 
Integration  Center  (ARCIC)  sponsored  a  number  of  pilot  tests  supporting  the  Connecting 
Soldiers  to  Digital  Applications  (CSDA)  initiative.  These  pilot  tests  focused  on  examining  the 
utility  of  employing  mobile,  handheld  technologies  and  gaming  solutions  to  bridge  the  gap 
between  Soldier  learning  and  training  as  a  function  of  time,  resources,  and  instructor  contact 
ratios. 


While  metrics  and  strategies  were  developed  to  assess  the  utility  of  each  of  these  pilot 
efforts,  there  was  no  plan  in  place  to  build  upon  these  largely  technological  assessments  to 
provide  practical  guidance  for  Advanced  Individual  Training  (AIT)  commanders  and  training 
developers  considering  the  development  and  integration  of  these  applications  and  technologies  in 
their  training  programs.  Such  guidance,  based  upon  lessons  learned  from  these  pilot  efforts  and 
best  practices  from  blended  learning  theory  and  practices,  would  provide  training  commanders 
and  developers  with  a  sound  foundation  for  determining  how  best  to  employ  these  emerging 
technologies  within  their  training  programs. 

This  research  was  conducted  to  create  a  decision  guide  to  aid  decision  makers  in 
designing  and  implementing  digital  applications  and  mobile  devices  into  AIT.  A  literature 
review  was  conducted  to  identify  best  practices  and  lessons  learned  from  previous  research. 
Additionally,  a  data  collection  was  conducted  to  build  on  the  findings  of  the  literature  review  and 
identify  the  factors  that  are  important  to  consider  in  designing  and  implementing  digital 
applications  and  mobile  devices  in  AIT.  Finally,  the  product  of  the  literature  review  and  data 
collection  yielded  a  comprehensive  decision  guide  that  was  evaluated  by  subject  matter  experts 
to  test  its  quality  and  make  additional  refinements. 

Literature  Review 

The  objective  of  this  literature  review  was  to  identify  existing  and  emerging  learning  and, 
more  appropriately,  training  theories,  strategies,  and  practices  related  to  integrating  mobile 
learning  technology  into  AIT.  Literature  on  learning  and  training  theories  and  mobile  device 
training  lessons  learned  was  used  as  a  foundation  for  developing  a  practical  decision  guide  for 
AIT  commanders  to  determine  how  and  when  to  most  effectively  integrate  digital  applications 
and  mobile  technologies  into  training. 

Military  and  nonmilitary  resources  were  identified  in  this  review.  Defense  Technical 
Infonnation  Center  (DTIC),  PsycINFO,  and  Google  Scholar  were  used  to  identify  resources. 
Additionally,  the  US  Army  Training  Concept  2012-2020  (Pam  525-8-3,  2011),  and  US  Army 
Learning  Model  (ALM)  (PAM  525-8-2)  were  reviewed  to  describe  related  aspects  of  the 
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emerging  Anny  training  environment,  and  the  role  of  mobile  learning  within  the  future  training 
context.  Conference  proceedings  from  the  Interservice,  International  Training,  Simulation  and 
Education  Conference  from  1990-2010  and  from  the  World  Conference  on  Mobile  and 
Contextual  Learning  (2001-2011)  were  also  reviewed. 

The  need  for  an  integrated  training  environment.  The  operational  environment  (OE) 
and  ARFORGEN  cycle  presents  challenges  that  must  be  understood  in  order  to  develop  more 
effective  training  methods.  The  Army  Training  Concept  (2011)  described  the  challenges  and 
respective  actions  that  must  be  taken  to  train  Soldiers  to  conduct  Full  Spectrum  Operations  in  an 
era  of  persistent  conflict.  The  state  of  persistent  conflict  and  operational  tempo  presents  a 
challenge  to  effectively  and  efficiently  train  during  the  reset  phase  of  ARFORGEN. 

This  challenge  is  exacerbated  by  the  lack  of  an  integrated  training  environment  (ITE). 

The  Army  Training  Concept  (2011)  identified  ITE  as  the  “Linkage  of  selected  training  aids, 
devices,  simulators,  and  simulations  (TADSS),  infrastructure,  mission  command  systems  (MCS), 
and  knowledge  management  systems,  and  a  training  framework  to  approximate  the  conditions  of 
the  operational  environment  for  training  and  education  for  full-spectrum  operations  in 
operational,  institutional,  and  self-development  training  domains”  (p.  96).  The  overarching  goal 
of  ITE  is  to  integrate  these  domains.  An  ITE  is  a  means  to  overcome  challenges  posed  by  the 
OE  and  an  aggressive  training  timetable.  In  an  ITE,  training  requirements  and  objectives  are  met 
by  effectively  connecting  the  Anny  learning  domains  to  form  a  network  and  then  connecting 
leaders  to  that  network.  The  institutions  become  learner-centric  by  developing  training  that  can 
be  accessed  by  Soldiers  when  they  need  it  (PAM  525-8-3,  2011). 

According  to  the  Army  Training  Concept  (2011),  the  integration  of  the  training  domains 
can  be  achieved  through  effective  development  and  implementation  of  resources.  These 
resources  should  be  easily  accessible  to  Soldiers  and  create  a  seamless  network  across  training 
domains.  Specifically,  mobile  learning  devices  were  identified  as  a  resource  capable  of  meeting 
OE  challenges  and  fulfilling  training  objectives  (PAM  525-8-3,  2011). 

The  Army  Training  Concept  described  the  Anny’s  vision  for  the  future  training  context. 
AIT,  as  a  part  of  institutional  training,  has  needed  to  continue  working  to  adapt  to  this  context  in 
order  to  meet  the  Army’s  overarching  goals  in  a  more  learner-centric  environment.  The  role  of 
mobile  devices  in  AIT  within  this  evolving  training  context  is  discussed  in  the  following  section. 

Mobile  learning  within  the  Army’s  future  training  context.  The  Anny  Training 
Concept  described  the  potential  for  mobile  training  devices  to  provide  instrumental  support  in 
creating  an  ITE.  The  basic  rationales  for  using  mobile  learning  devices  in  training  are  access  and 
portability.  First,  mobile  devices  provide  a  link  between  the  operational,  self-development,  and 
institutional  domains.  PAM  525-8-3  (2011,  p,  101)  stated  that  mobile  training  devices  “provide 
access  to  learning  content,  courseware,  and  career  data,  as  well  as  perfonnance  support 
applications;  can  be  used  to  transmit  information  that  must  be  captured,  analyzed,  and  important 
lessons  rapidly  disseminated  to  those  who  need  to  know  and  can  take  action”  (PAM  525-8-3, 

201 1,  p.  101).  The  centers  of  excellence  (CoEs)  and  schools  are  hubs  for  information  and 
training.  By  integrating  mobile  training  devices  within  their  training  environments,  Anny 
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Soldiers  and  civilians  will  be  able  to  directly  access  Anny  networks  to  meet  training  needs  (see 
PAM  525-8-3). 

The  second  basic  attribute  of  mobile  training  devices  is  portability  as  mobile  devices 
allow  learners  to  not  be  bound  by  location  (PAM  525-8-3,  2011).  For  example,  Soldiers  can  use 
the  infonnation  or  data  accessed  through  these  devices  as  a  reference  or  tool  in  conducting 
tactical  operations,  completing  administrative  actions,  enhancing  their  individual  knowledge  and 
skills  on  specific  tasks,  or  managing  their  careers.  Most  importantly  for  AIT  environments, 
Soldiers  and  instructors  can  use  mobile  learning  devices  to  enhance  the  student  centered  aspects 
of  classroom  training,  provide  a  means  to  fully  exploit  training  time  for  every  Soldier,  and  better 
connect  classroom  instruction  to  field  training. 

Mobile  learning  can  be  an  effective  method  for  delivering  training  in  AIT  and  can  help  to 
create  an  ITE.  To  fulfill  their  role  in  the  future  Army  Training  Concept,  the  CoEs  and  schools 
need  to  appropriately  translate  curricula  and  resources  to  a  mobile  learning  environment  and 
augment  training  with  mobile  instruction.  Thus,  this  literature  review  considered  the 
implementation  of  mobile  learning  in  AIT  and  future  training  contexts  by  addressing  the 
following  questions  concerning  the  implementation  of  mobile  learning  in  AIT. 

•  What  are  the  AIT  learning  environment  features  that  are  most  critical  to  AIT  training 
design? 

•  How  can  blended  learning  theories  be  applied  to  understand  the  design  components 
related  to  AIT  training? 

•  What  learning  theories  and  instructional  practices  are  most  relevant  to  AIT  training  and 
how  could  they  be  used  to  enhance  AIT  training? 

•  What  are  the  best  practices  and  lessons  learned  for  designing,  developing,  and 
implementing  AIT  training  using  mobile  devices? 

AIT  training  context.  It  is  valuable  to  acknowledge  distinctions  between  general 
education,  professional  development,  and  technical  training  when  examining  AIT  training 
content.  As  Bonk  and  Wisher  (2000)  pointed  out,  there  are  “fundamental  differences  in  the 
goals,  outcomes,  and  eventual  application  of  the  underlying  instruction”  (p.  3)  between  education 
and  training.  “AIT  continues  Soldier  development  along  with  basic  competencies  for  military 
occupational  specialty  (MOS)  skills.  It  is  often  considered  as  a  trade  school  where  learning 
individual  basic  job  skill  sets  are  acquired”  (TRADOC  Pam  525-8-2,  201 1,  p.  46).  While  AIT 
includes  the  ongoing  development  of  Soldier  values  and  general  skills,  the  primary  focus  of  AIT 
is  on  MOS  specific  technical  skill,  as  part  of  the  Army’s  integrated  training  strategy.  AIT 
schools  vary  in  length  from  six  weeks  to  over  1  year,  and  comprise  a  vast  number  of  different 
occupations,  training  content,  training  objectives,  training  contexts,  equipment,  and  practice 
opportunities.  Therefore,  special  consideration  must  be  given  to  detennine  optimal  uses  for 
mobile  learning  technology  across  such  a  wide  array  of  training  situations. 

In  addition  to  the  focus  on  task-specific  skills,  AIT  is  limited  in  time  and  relies  heavily 
upon  an  institutional  environment  and,  more  often  than  not,  on  Soldiers  interacting  with  highly 
technical  equipment,  systems,  and  environments.  The  goal  of  AIT  instruction  is  much  more 
similar  to  mastery  learning  than  general  knowledge  education  and  professional  development. 
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Mastery  learning  is  based  upon  the  notion  that  learning  is  a  function  of  time  (Bloom,  1974)  and 
based  upon  some  criterion  of  learning  (Carroll,  1963).  AIT  courses  typically,  if  not  universally, 
require  criterion-referenced  proficiency  tests  to  ensure  skills  are  being  learned,  which  can  be 
applied  under  standardized  conditions  in  some  observable,  quantifiable  way  to  an  established 
standard.  Ensuring  proficiency  to  an  established  minimum  level  is  a  critical  component  of  any 
position  that  involves  a  high  danger  element  (e.g.,  tank  gunner,  medical  professional,  infantry 
Soldier,  etc.). 

When  examining  AIT,  it  is  also  valuable  to  consider  different  MOSs.  All  MOSs  require 
certain  knowledge  and  skills  to  perfonn  effectively.  However  some  MOSs  have  significant 
physical  and  psychomotor  aspects  that  are  not  required  of  other  MOSs.  For  example,  vehicle, 
weapon,  or  health  care  specialists  must  perfonn  physical  skills  to  a  level  of  proficiency  on  real 
equipment  that  other  MOSs  may  not.  During  AIT,  these  MOSs  require  proficiency  tests  on  the 
actual  equipment  (or  very  high  fidelity  simulations).  Other  MOSs  that  emphasize  using 
primarily  cognitive  and  infonnation  processing  skills  such  as  intelligence,  information 
technology,  and  other  service  and  support  functions  may  not  require  as  much  time  spent  on 
qualifying  equipment.  Therefore,  the  nature  of  the  content  and  the  interaction  with  equipment 
may  limit  the  usefulness  of  mobile  devices  for  certain  training  content. 

Several  categories  of  learning  theories  and  learning  and  instructional  practices  are 
especially  relevant  for  considering  effective  implementation  of  mobile  devices  in  AIT.  These 
theories  and  practices  are  reviewed  in  the  following  sections. 

Learning  Theories 

Learning  theories  represent  a  wide  array  of  ideas,  models,  and  concepts  related  to  how 
children  (pedagogy)  and  adults  (andragogy)  acquire  knowledge  and  skills.  There  is  no  consensus 
on  which  learning  theories  are  the  most  robust  or  relevant  to  specific  types  of  learning.  Though 
no  one  theory  stands  above  the  others,  all  of  the  learning  theories  discussed  herein  provide 
potentially  valuable  methods  for  executing  effective  training.  Thus,  knowledge  of  the  learning 
theories  can  be  exploited  to  apply  aspects  of  each  theory  in  the  design  of  effective  training  with 
mobile  devices. 

An  understanding  of  the  distinction  between  learning  theories  and  instructional  practices 
is  important  in  interpreting  the  role  of  a  learning  theory  in  determining  the  utility  of  mobile 
learning  and  integration  of  mobile  technology  into  AIT.  Learning  theories  establish  the 
foundation  for  why  an  instructional  practice  may  or  may  not  be  effective.  For  example,  operant 
conditioning,  a  theory  of  behaviorism,  would  posit  that  instructional  practices  that  provide  a 
timely  reinforcement  for  appropriate  student  actions  in  response  to  certain  stimuli  would  be  more 
successful  than  instructional  practices  that  do  not  provide  such  reinforcement.  While  the  theory 
may  support  specific  practices  within  the  instructional  system,  it  is  unlikely  to  be  the  sole  source 
for  the  development  of  training  content  related  to  a  course  of  instruction.  In  reality,  several,  if 
not  most,  learning  theories  have  some  relevancy  to  the  traditional  systems  approach  to  training 
and  military  training  course  design.  This  review  attempts  to  examine  those  learning  theories  that 
are  most  pertinent  to  improving  learning  outcomes  related  to  mobile  and  blended  learning 
applications  and  technologies  in  technical  training. 
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General  learning  theories.  The  following  theories  and  practices  were  selected  for 
review  based  upon  their  contribution  to  explaining  various  aspects  of  learner  motivation  and 
reinforcement,  learning  content  and  context,  and  the  learning  process.  General  theories  are  first 
described  and  then  more  specific  instructional  practices  related  to  the  use  of  blended  and  mobile 
learning  techniques  are  provided.  Four  prevailing  theories1  of  learning  are  described: 
behaviorism,  cognitivism,  constructivism,  and  andragogy.  In  some  cases,  instructional  practices 
may  be  closely  associated  with  certain  learning  theories,  such  as  self-directed  learning  and 
andragogy,  however,  in  many  cases  these  practices  may  support  multiple  theories. 

Behaviorism  and  operant  conditioning.  Behaviorism  as  a  learning  theory  has  a  long 
and  rich  history  incorporating  perspectives  from  Pavlov,  Watson,  Hull,  Spence,  and  most  notably 
Skinner  (see  Moore  [2011]  for  a  recent  overview).  For  the  sake  of  brevity,  this  review  focuses 
on  the  role  of  operant  conditioning  in  learning  and  training.  Operant  conditioning  can  be 
extended  to  learning  theory  in  terms  of  the  reinforcement  of  learning  behaviors.  Simply  stated, 
operant  conditioning  maintains  that  behaviors  that  are  reinforced  will  be  associated  with  other, 
conditional  stimuli.  Operant  conditioning  is  distinguished  from  classical  conditioning  in  that  it 
deals  with  voluntary  behavior.  However,  even  classical  conditioning  is  relevant  for 
understanding  certain  aspects  of  military  training,  particularly  motor  skills  development  (Lee  & 
Schmidt,  2005).  Coaches  and  athletes  often  refer  to  muscle  memory,  which  develops  with 
repeated  physical  motion  of  patterned  actions.  The  connection  between  stimulus  and  action  is 
often  made  in  milliseconds  and  thought  is  nearly  eliminated.  As  the  repetition  and  reinforcement 
of  these  actions  leads  to  increased  proficiency,  confidence  also  builds. 

Both  operant  and  classical  conditioning  could  be  considered  in  the  application  of  mobile 
devices  in  Army  training,  as  they  serve  to  more  efficiently  connect  the  Soldier  to  the  stimuli, 
responses  and  reinforcement  associated  with  trained  behaviors  and  actions.  Among  the  issues 
related  to  mobile  devices  for  which  operant  and  classical  conditioning  learning  theories  are 
applicable  are  the  ability  of  the  devices  to  provide  the  necessary  cues  for  proper  student 
response,  the  ability  of  the  devices  to  provide  appropriate  and  timely  reinforcement  of  actions, 
and  the  ability  of  the  devices  to  provide  cues  related  to  improper  behaviors  or  actions  (see 
Naismith,  et  ah,  2004  for  more  details).  In  addition,  the  opportunities  for  extended  practice  and 
instructor  independent  (as  well  as  integrated)  reinforcement  seemingly  represent  considerable 
value-added  capabilities  from  the  application  of  mobile  devices  within  AIT  environments  from 
these  learning  theory  perspectives. 

Cognitivism.  Cognitivism,  as  a  learning  theory,  involves  a  broad  set  of  sub-theories  that 
consider  human  thought  processes  and  the  functioning  of  memory,  information  processing,  and 
knowledge  fonnation  as  paralleling  that  of  computer  processing.  Mayer’s  theory  of  multimedia 
learning  (Mayer  &  Johnson,  2008)  could  have  importance  to  the  design  of  learning  applications, 
as  it  advocated  replacing  visual  text  with  spoken  text  (the  modality  effect)  and  adding  visual  cues 
relating  elements  of  a  picture  to  the  text  (the  cueing  effect)  to  increase  the  effectiveness  of 


1  Connectivism  (Seimens,  2005;  Downes,  2005)  has  been  proposed  as  a  new  theory  of  learning  particularly  suited 
for  determining  the  role  of  technology  in  learning.  However,  the  lack  of  significant  research  or  even  specific 
practical  applications  related  to  Connectivism  relegate  it  to  more  of  a  learning  perspective  than  a  theory  for 
discussion  in  this  review. 
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multimedia  instructions  in  terms  of  better  learning  results  or  less  mental  effort  spent.  A  recent 
meta-analysis  by  Adesope  and  Nesbit  (2011)  demonstrated  that  students  receiving  spoken- 
written  and  written-only  presentations  did  not  exhibit  performance  differences,  but  students  who 
experienced  spoken- written  presentations  did  outperform  those  that  received  spoken-only 
presentations.  Mayer’s  theory  of  multimedia  learning  and  the  results  of  Adesope  and  Nesbit’s 
meta-analysis  suggest  that  written  presentation  of  learning  material  combined  with  spoken 
instruction  can  increase  the  effectiveness  of  instruction,  specifically  in  comparison  to  types  of 
instruction  that  rely  solely  on  spoken  instruction.  These  findings  are  relevant  to  AIT  instruction 
in  two  ways.  First,  applications  should  not  rely  solely  on  spoken  forms  of  instruction  or  audio  to 
deliver  content,  but  rather  content  should  include  both  spoken  and  written  forms  of  instruction. 
Second,  mobile  devices  could  aid  in  AIT  training  settings  that  typically  rely  solely  on  verbal  or 
spoken  forms  of  instruction  by  providing  supplemental  text  to  accompany  and  support  the  verbal 
and  spoken  instruction. 

The  application  of  cognitivism  to  training  is  intended  to  reduce  the  mental  load  on 
students  and  assist  in  organizing  the  knowledge  and  skills  to  be  learned  through  the  use  of 
multiple  sources  of  information,  but  not  such  that  they  overwhelm  the  processing  capacity  of  the 
individual.  There  are  several  instructional  practices  that  explore  different  types  of  cognitive 
strategies  and  strategies  that  engage  the  participant  in  active  learning,  such  as  learning  from 
others  (Gibbs,  1999),  tutoring  others  (Merrill,  Reiser,  Merrill,  &  Landes,  1995),  self-regulation 
(Zimmerman,  1989),  instructional  scaffolding  (Vygotsky,  1978).  Mobile  devices  provide  an 
additional  media  source  for  students  to  search,  receive,  and  experience  learning  content,  practice 
methods,  and  feedback  on  performance.  The  applicability  of  cognitivism  to  the  use  of  mobile 
devices  in  AIT  is  pervasive  and  complex,  and  should  be  considered  early  in  the  media  selection 
process  such  that  device  use  is  consistent  with  learner  background,  capabilities,  and  content 
integration  requirements. 

Constructivism.  Constructivist  theories  propose  that  knowledge  is  actively  constructed 
by  the  individual  learner  and  knowing  is  an  active  and  adaptive  process  that  serves  to  organize 
the  individual’s  experiential  world  (Mayer,  1992;  Hendry,  1996).  Learners  bring  their  past 
experiences  to  the  learning  setting  which  establishes  a  baseline  for  gaining  more  knowledge  and 
building  upon  this  baseline.  Constructivism  asserts  that  the  role  of  learning  (and  training)  is  to 
allow  the  learner  to  discover  how  additional  knowledge  and  experience  fits  into  their  existing 
cognitive  sets  or  mental  representations  of  reality.  Constructivism  is  associated  with  many 
instructional  practices,  including  self  directed  and  self  regulated  learning,  collaborative  learning, 
and  goal  setting.  Two  principal  considerations  regarding  the  application  of  constructivism  and 
the  use  of  mobile  devices  are  implied.  First,  the  way  in  which  instructors  use  mobile  devices  to 
facilitate  construction  of  learning  by  the  learner  is  important  as  particular  approaches  may  foster 
or  hinder  various  goals,  and  it  is  important  to  assess  which  approaches  work  best  for  which 
goals.  Second,  the  degree  to  which  learners  can  use  the  mobile  device  to  structure,  re-organize, 
and  experience  emerging  learning  content  will  also  aid  in  the  long  tenn  use  of  the  devices  and 
transfer  of  knowledge  to  job  tasks  and  performance  contexts. 

Andragogy.  Andragogy,  most  often  associated  with  Malcolm  Knowles,  refers  to  adult 
learning  principles  and  more  specifically  to  the  notion  that  adult  learners  prefer  learning  methods 
that  allow  them  autonomy  and  the  ability  to  direct  their  own  learning  activities  (Hiemstra,  1994). 
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Andragogy  stipulates  that:  (1)  adults  need  to  know  why  they  need  to  learn  something,  (2)  adults 
need  to  leam  experientially,  (3)  adults  approach  learning  as  problem-solving,  and  (4)  adults  learn 
best  when  the  topic  is  of  immediate  value.  In  practical  tenns,  andragogy  means  that  instruction 
for  adults  needs  to  focus  more  on  the  process  and  less  on  the  content  being  taught.  Strategies 
such  as  case  studies,  role  playing,  simulations,  and  self  evaluation  have  been  found  as  most 
useful  with  adult  learning.  The  importance  of  andragogy  as  a  theory  for  the  implementation  of 
mobile  and  digital  devices  into  Army  training  is  the  emphasis  on  the  learner  to  self-direct,  self- 
manage,  and  self-motivate  toward  effective  learning  content  and  practice  (Garrison,  1997). 

These  principles  are  reflected  in  the  Anny’s  self-development  pillar  of  the  education  process. 

Andragogy,  self-directed  learning,  and  other  self-learning  motivation  theories  have 
extensive  implications  related  to  the  use  of  mobile  devices  in  all  Army  environments,  given 
ALM  and  its  implications  for  extending  training  and  learning  beyond  the  classroom  and  over  a 
Soldier’s  entire  career.  The  degree  to  which  mobile  devices  will  be  used  for  learning  purposes, 
given  these  devices  allow  individuals  to  choose  when,  where,  and  even  how  they  want  to  leam, 
is  controlled  by  the  individual.  Therefore,  it  will  be  important  to  recognize  individual 
differences  in  motivation,  understanding,  and  rewards  related  to  self-learning  and  the  ability  to 
direct  one’s  own  learning.  It  will  not  simply  be  enough  to  train  Soldiers  on  the  use  of  the  devices 
without  having  some  information  related  to  the  potential  impact  that  lack  of  motivation  and 
intrinsic  motivation  play  in  individual  learning  effectiveness. 

Learning  and  Instructional  Practices 

Perhaps  more  relevant  to  the  application  of  mobile  and  digital  devices  to  AIT  than 
learning  theories  are  the  instructional  practices  related  to  these  theories  and  the  research  on  their 
effectiveness  and  applicability  to  technical  training  (or  at  a  minimum  work-related  learning). 

The  following  instructional  practices,  some  of  which  are  often  described  as  theories  themselves, 
were  examined  for  relevancy  to  AIT  and  the  use  of  mobile  technology. 

Blended  learning.  In  an  ARI  report,  Tucker  (2010)  identified  several  potential 
approaches  to  mobile  learning  for  Anny  institutional  training.  Tucker’s  purpose  was  to  identify 
literature  that  would  be  helpful  for  incorporating  smartphones  into  Anny  training.  Tucker 
suggested  that  a  blended  approach  should  be  taken  to  better  exploit  the  advantages  of  mobile 
learning  within  institutional  training.  A  blended  approach  would  add  to  the  value  of  current 
instruction  by  applying  mobile  learning  pedagogies/andragogies  and  leveraging  the  capabilities 
and  tools  of  mobile  devices  (e.g.  applications,  camera). 

Blended,  hybrid,  and  e-leaming  have  been  described  as  referring  to  an  evolving  set  of 
definitions  related  to  a  combination  of:  face-to-face  and  online  instruction,  media  and 
technologies  and  pedagogical  methodologies  (Sharma,  2010).  Regardless  of  the  components  of 
blended  learning,  Zhao  (2008)  recommended  that  blended  e-leaming  should  include  well- 
designed  and  optimized  learning  delivery  methods,  as  well  as  specific  objectives,  learning 
events,  tasks,  and  purposes.  Seemingly,  any  instructional  system  that  includes  methods  other 
than  classroom-based,  instructor-led  components  may  satisfy  the  definition  for  blended  learning. 
Further,  classroom-based,  instructor-led  training  can  be  combined  with  online  and  technological 
components  to  create  a  blended  learning  experience. 
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Yoon  and  Lim  (2007)  provided  an  overview  of  blended  learning  research  findings  and 
offered  a  conceptual  framework  of  blended  learning  that  included  a  strategic  focus  on  learning 
objectives  and  the  available  technology.  Tsai  (2011)  proposed  and  found  attitudinal  support  for 
a  model  of  hybrid  e-leaming  that  contained  elements  of  scenario  (realistic  examples), 
collaborative  (student-to-student),  and  scaffolding  (supported  by  others)  aspects. 

Carman  (2005)  applied  constructivist  (i.e.,  knowledge  is  actively  constructed),  cognitivist 
(i.e.,  learning  involving  human  thought  processes),  and  perfonnance  support  learning  theories  to 
identify  the  following  five  key  ingredients  to  blended  learning  design. 

1.  Live  Events:  Synchronous,  instructor-led  learning  events  in  which  all  learners  participate 
at  the  same  time,  such  as  in  a  live  “virtual  classroom.” 

2.  Online  Content:  Learning  experiences  that  the  learner  completes  individually,  at  his  or 
her  own  speed  and  on  his  or  her  own  time,  such  as  interactive,  Internet-based  or  CD- 
ROM  training. 

3.  Collaboration:  Environments  in  which  learners  communicate  with  others,  for  example, 
through  e-mail,  threaded  discussions,  and  online  chatting. 

4.  Assessment:  A  measure  of  learners’  knowledge.  Pre-assessments  can  come  before  live  or 
self-paced  events,  to  detennine  prior  knowledge,  and  post-assessments  can  occur 
following  scheduled  or  online  learning  events,  to  measure  learning  transfer. 

5.  Reference  Materials:  On-the-job  reference  materials  that  enhance  learning  retention  and 
transfer,  including  checklists,  diagrams,  decision  aids,  etc. 

The  empirical  literature  on  blended  learning  environments  has  supported  the 
effectiveness  of  blended  learning  in  some  domains.  Improved  learning  outcomes  under  blended 
learning  conditions  (online  with  traditional  classroom)  have  been  demonstrated  by  Boyle, 
Bradley,  Chalk,  Jones  and  Pickard  (2003)  and  Dowling,  Godfrey,  and  Gyles  (2003).  In  addition, 
Starenko,  Vignare,  and  Humbert  (2007)  described  the  results  of  an  earlier  study  showing  student 
satisfaction  increased  under  blended  learning  conditions.  It  is  important  to  note  that  e-learning 
has  also  been  associated  with  negative  consequences,  including  a  steep  learning  curve  for  the 
technology  involved  and  procrastination  with  asynchronous  learning  (Yoon  and  Lim  (2007). 
Results  regarding  strictly  online  or  distance  learning  courses  without  blended  components  are 
mixed.  For  example,  Reasons,  Valadares,  and  Slavkin  (2005)  found  that  students  participating  in 
an  online  course  outperfonned  those  in  both  classroom  only  and  blended  (classroom  and  online) 
conditions.  However,  Stein  (2004)  found  that  more  important  than  whether  or  not  blending  was 
a  component  of  a  course  was  the  level  of  structure  related  to  objectives,  assignments,  deadlines, 
and  teacher  expectations  of  dialogue  and  student  interaction. 

Hsu  and  Hsieh  (2011)  conducted  a  cross  sectional  study  of  learner  demographics  (age, 
previous  school,  months  working,  class  rank),  learning  behaviors  (time  spent  online  and  number 
of  web  talks),  and  self  reported  learning  performance  (case  analysis  attitude  scale  [CAAS],  case 
analysis  self-evaluation  scale  [CASS],  metacognition  scale  [MS],  and  blended  learning 
satisfaction  scale  [BLSS])  with  ethics  course  scores.  Frequency  of  online  dialogues,  time  spent 
on  the  internet  in  minutes,  and  CAAS  score  were  found  to  be  positive,  significant  predictors  of 
learning  outcome.  The  learning  outcome  was  the  final  course  score  that  was  comprised  of 
overall  perfonnance  in  online  scenario  discussion,  classroom  participation,  team  debate,  and 
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group  report.  Findings  imply  that  both  time  and  attitudes  toward  the  learning  method  affect 
learning  in  blended  courses. 

The  applicability  of  blended  learning  theories  and  practices  to  the  use  of  mobile  devices 
in  AIT  essentially  involves  when  and  where  the  devices  will  be  used.  That  is,  will  they  be  used 
for  in-class  only  purposes  or  will  they  be  used  by  students  outside  the  classroom,  including 
breaks  during  the  workday,  during  leisure  time,  and  possibly  even  to  access  other  institutional 
training  when  remote  from  the  training  location?  In  determining  the  utility  of  using  mobile 
devices  in  blended  learning,  it  is  important  to  consider  learner  familiarity  and  other  attitudes 
toward  the  devices,  instructor  attitudes,  and  how  well  learning  objectives  are  structured  into  the 
learning  activities. 

Mobile  learning.  Although  somewhat  dated  for  a  quickly  evolving  field,  Naismith, 
Lonsdale,  Vavoula,  and  Sharpies  (2004)  provided  an  extensive  literature  review  on  the  learning 
theories  and  empirical  research  related  to  mobile  technologies  and  learning  that  is  relevant  for 
the  current  effort.  The  authors  identified  the  following  as  key  considerations  when  integrating 
mobile  technologies  into  learning  and  training: 

•  Context  -  the  learner’s  desire  for  anonymity  and  privacy  may  affect  learning 

effectiveness, 

•  Mobility  -  consideration  of  the  learner’s  ability  to  engage  in  activities  not  associated  with 

the  learning  objectives, 

•  Learning  over  time  -  consideration  of  the  how  learning  will  be  measured  and  recorded  in 

mobile  environments, 

•  Informality  -  students  may  perceive  the  use  of  technology  to  threaten  their  existing  social 

networks,  and 

•  Ownership  -  students  may  want  more  control  over  their  devices  in  the  classroom  than 

feasible  during  learning  and  training. 

O’Malley,  Vavoula,  Glew,  Taylor,  Sharpies,  and  Lefrere  (2003)  provided  guidelines  for 
learning  in  mobile  environments  that  included  an  overview  of  learning  theories  and  their 
influence  on  learning  technologies.  The  authors  provided  examples  of  specific  technology 
applications  under  behaviorist,  constructivist,  situated,  collaborative,  informal/lifelong,  and 
instructor  supported  theoretical  approaches.  A  set  of  27  guidelines  were  then  identified  that 
include  cost,  usability,  input,  output,  learner  involvement,  roles,  and  other  aspects  that  assist 
users  in  applying  mobile  devices  to  improve  learning  outcomes.  Each  of  the  guidelines  is 
pertinent  to  the  integration  of  mobile  devices  into  Army  AIT  and  were  considered  as  appropriate 
for  input  into  the  final  product. 

A  recent  article  by  Keskin  and  Metcalf  (2011)  proposed  15  different  theories  as 
applicable  to  mobile  learning  and  provided  definitions,  focus,  and  examples  with  mobile 
technologies  in  support  of  their  proposal.  However,  it  is  debatable  as  to  whether  what  the 
authors  identified  as  theories  (e.g.,  context  awareness  learning,  collaborative  learning,  informal 
learning)  may  be  better  categorized  as  instructional  practices  or  strategies  rather  than  theories. 
The  authors  did  not  provide  any  specific  research  supportive  of  these  theories  and  how  they  were 
applicable  to  integrating  mobility  into  learning  practices. 
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Sharpies,  Taylor,  Vavoula  (2005)  presented  a  framework  for  analyzing  the  relationship 
between  mobile  technology  and  learning  as  an  adaptation  of  Engestrom’s  expansive  activity 
model  (Engestrom,  1987).  This  framework  posited  learning  as  an  activity  system  that  both 
supports  and  constrains  the  learner  in  their  development.  A  semiotic  layer  consists  of  learner 
object  oriented  actions  that  are  mediated  by  cultural  tools  and  signs  (language,  etc.).  A 
technological  layer  consisting  of  tools  and  devices  serves  to  aid  learners  in  making  connections, 
aiding  recall,  and  reflection.  Sharpies  et  al.  (2005)  noted  that  “Learning  occurs  as  a  socio¬ 
cultural  system,  within  which  many  learners  interact  to  create  a  collective  activity  framed  by 
cultural  constraints  and  historical  practices”  (p.  7). 

Yen  and  Lee  (2011)  used  classroom  instruction,  web-based,  and  mobile  learning 
scenarios  on  a  problem  solving  exercise  and  identified  three  types  of  learners  through  content 
and  cluster  analysis  of  student  learning  process  data:  a  hybrid-oriented  group,  a  technology- 
oriented  group,  and  an  efficiency-oriented  group.  Learners  in  the  hybrid-oriented  group  used  the 
classroom,  mobile,  and  web  scenarios  approximately  equally  and  followed  instructor’s  teaching 
procedures  passively.  Students  in  the  technology-oriented  group  spent  the  majority  of  their  time 
using  mobile  and  web  technologies  but  they  displayed  lower  problem  solving  abilities  including 
a  lack  of  understanding  and  planning  difficulties.  The  efficiency-oriented  group  perfonned 
better  on  problem  solving  tasks  than  the  other  groups  and  spent  more  time  on  the  assigned  task 
and  time  monitoring  the  learning  process.  This  research  has  implications  for  examining 
individual  differences  in  understanding  and  use  of  blended  technologies. 

Mobile  learning  theories  are  important  to  consider  in  determining  the  applicability  of 
mobile  devices  in  AIT  as  they  specify  student,  training  context,  mobile  device,  and  training 
objective  characteristics  to  consider  when  evaluating  mobile  device  utility  in  a  particular  context. 
Many  of  the  value  aspects  related  to  mobile  learning  theories  have  already  been  incorporated  in 
the  Army  Learning  Model  from  a  general  utility  perspective.  However,  decisions  related  to  the 
particular  utility  of  a  device  to  a  specific  course  or  objective  will  benefit  from  close  examination 
of  these  characteristics  as  they  relate  to  the  specific  learning  context.  Lor  example,  some  training 
objectives  related  to  weapon  firing  accuracy  may  not  benefit  from  the  use  of  mobile  devices 
because  they  do  not  provide  the  necessary  haptic  fidelity  related  to  trigger  or  firing  mechanisms. 
Yet,  a  mobile  device  may  be  valuable  for  other  objectives  related  to  weapon  sighting  or  support 
position. 

Self  directed  learning  and  self  regulated  learning.  Knowles  (1975)  defined  self- 
directed  learning  (SDL)  as  “a  process  in  which  individuals  take  the  initiative,  with  or  without  the 
help  from  others,  in  diagnosing  their  learning  needs,  formulating  goals,  identifying  human  and 
material  resources,  choosing  and  implementing  appropriate  learning  strategies,  and  evaluating 
learning  outcomes”  (p.  18).  There  has  been  extensive  interest  in  SDL  and  considerable  attention 
given  to  measuring  self-directed  learner  readiness  (Gugliehnino,  1978;  1989).  However,  the 
empirical  findings  regarding  the  effects  of  SDL  on  learning  outcomes  are  somewhat  mixed 
(Chou  &  Chen,  2008,  as  reported  in  Deyo,  Huynh,  Rochester,  Sturpe,  &  Kiser,  2011). 

Maurer,  Weiss,  and  Barbeite  (2003)  found  that  individuals  who  are  oriented  toward  their 
own  professional  development  tend  to: 
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•  Have  prior  experience  with  development  in  the  same  area, 

•  Perceive  themselves  as  possessing  the  qualities  needed  for  learning, 

•  Have  social  support  for  development  at  work, 

•  Are  involved  in  their  jobs, 

•  Have  insight  into  their  career, 

•  Believe  in  the  need  for  self  development,  and 

•  Believe  in  their  abilities  to  develop  skills  and  receive  benefits  from  development. 

Murad,  Coto-Yglesias,  Varkey,  Prokop,  and  Murad  (2010)  conducted  a  meta-analysis  of 
59  studies  that  compared  SDL  with  traditional  teaching  methods  for  health  professionals.  The 
results  revealed  that  the  use  of  SDL  was  associated  with  a  moderate  improvement  in  knowledge- 
based  outcomes  compared  with  didactic  instruction,  but  there  were  no  significant  differences 
between  the  two  with  respect  to  skill  and  attitude-based  outcomes.  The  study  also  found  SDL  to 
be  more  effective  when  learners  were  involved  in  identifying  their  own  learning  resources. 

Closely  related  to  the  concept  of  SDL  is  that  of  self-regulated  learning  (SRL).  Self¬ 
regulation,  also  referred  to  as  metacognition  (self-regulation  is  sometimes  conceptualized  as 
broader  than  cognition)  is  defined  as  an  individual’s  awareness,  knowledge,  and  control  over  his 
or  her  cognitions  and  behaviors  (Flavell,  1979;  Martinez,  2006),  and  includes  planning, 
monitoring,  and  revising  behavior  (Brown,  Bransford,  Ferrara,  &  Campione,  1983).  SRL  theory 
posits  that  the  learner  manages  affective,  cognitive,  and  behavioral  processes  throughout  a 
learning  experience  to  reach  desired  goals.  A  recent  meta-analysis  of  the  self-regulated  literature 
specific  to  work-related  learning  (Sitzmann  &  Ely,  2011)  revealed  that  goal  level,  persistence, 
effort,  and  self-efficacy  were  the  most  significant  predictors  of  learning,  after  controlling  for  the 
effects  of  cognitive  ability  and  pre-training  knowledge  of  the  subject  matter.  Equally  as 
important  was  the  finding  that  planning,  monitoring,  help  seeking,  and  emotional  control  did  not 
provide  significant  contribution  to  the  prediction  of  learning  in  self-regulated  contexts. 

Loyens,  Magda,  and  Rikers  (2008)  examined  SDL  in  problem-based  learning  and 
compared  SDL  and  SRL  theories.  The  authors  proposed  the  primary  distinction  between  the  two 
is  the  role  of  the  learner  in  identifying  and  evaluating  the  reference  material  and  instructional 
content.  The  tenets  of  SDL  suggest  that  learners  are  responsible  for  identifying  what  is  to  be 
learned,  whereas  in  SRL  an  instructor  or  other  guide  could  identify  what  is  to  be  learned  (Loyens 
et  ah).  The  origin  of  the  learning  experience  is  of  less  significance  in  SRL,  compared  to  SDL. 
Self-regulation  focuses  more  on  the  individual’s  role  in  learning  after  “what  is  to  be  learned”  has 
been  detennined,  e.g.,  monitoring  their  progress,  detennining  when  they  are  having  problems, 
and  adjusting  their  learning  or  behaviors  accordingly.  As  such,  self-regulation  is  associated  with 
knowledge  acquisition,  skilled  performance  at  the  end  of  training,  and  self-efficacy  (Ford,  et  ah, 
1998).  These  factors  have  also  been  linked  with  effective  training  outcomes  (Colquitt,  LePine, 

&  Noe.,  2000;  Ford  et  ah,  1998;  Kozlowski  et  ah,  2001). 

SDL  and  SRL  have  broad  implications  for  integrating  mobile  devices  in  AIT.  Among  the 
issues  for  consideration  in  allowing  AIT  students  to  use  these  devices  will  be: 

1 .  Use  of  the  devices  for  Army  training  and  education  purposes  vs.  other  uses, 
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2.  Whether  use  will  be  guided  by  the  instructor  or  completely  self-directed, 

3.  Whether  the  user  will  monitor  and  evaluate  their  use  of  the  device  and  self-direction,  and 

4.  Individual  characteristics  of  the  learner,  including  self-efficacy,  experience  with  mobile, 

and  self-directed  learning,  and  readiness  for  self-direction. 

Instructional  designers  and  instructors  themselves  should  consider  these  issues  when 
identifying  optimal  opportunities,  objectives,  and  contexts  for  mobile  device  integration.  Prior 
assessments  of  student  capabilities,  motivation  to  use  devices,  tendencies  toward  self  directed 
learning,  and  the  specific  applications  of  the  devices  themselves  should  also  be  considered.  For 
example,  in  situations  where  students  may  not  be  assessed  as  highly  self-motivated  or  self- 
directed,  perhaps  networked  applications  of  the  devices  may  better  support  learning  objectives 
by  providing  additional  feedback  and  oversight  of  their  progress  than  more  independent  learning 
tasks  or  activities.  Alternatively,  where  students  appear  highly  self-motivated,  more  individual 
tasks  or  exercises  implemented  on  mobile  devices  providing  greater  opportunities  for 
autonomous  self-directed  learning  may  prove  more  effective. 

Goal  setting.  Common  to  both  SDL  and  SRL  theories  is  the  concept  of  goal  setting. 

Goal  setting  (Locke  &  Latham;  2002)  is  actually  a  motivational  theory,  but  can  be  considered  an 
important  practice  related  to  assisting  adult  learners  to  specify,  manage,  and  support  their  self¬ 
development.  The  acronym  SMART  (specific,  measurable,  achievable,  realistic,  and  time  based) 
has  been  used  to  represent  the  most  important  elements  of  successful  goal  setting.  Mobile 
technology  could  play  a  role  in  several  ways  to  support  goal  setting  and  self-directed  learning, 
including  providing  access  to  information  to  establish  goals  and  rewards,  serving  as  the  tool  for 
managing  and  tracking  goals,  and  providing  access  to  infonnation  necessary  to  achieve  goals. 

Problem-Based  and  Studio-Based  Learning.  Problem-Based  Learning  (PBL)  and 
Studio-Based  Learning  (SBL)  are  two  similar  techniques  of  scenario-based  learning  that  have 
received  considerable  attention  in  the  research  literature.  PBL  was  primarily  developed  in  the 
mid-1960s  as  a  useful  instructional  alternative  to  conventional  teaching.  PBL  is  an  instructional 
model  originally  developed  in  medical  schools,  in  which  students  are  given  a  complex  problem 
to  solve  that  may  not  have  a  single  correct  answer  (Hmelo-Silver,  2004).  The  teacher  acts  as  a 
facilitator  and  guides  the  learning  process  through  open-ended  questioning,  thus  promoting  self- 
directed  learning  and  facilitating  a  sense  of  intrinsic  motivation.  Hoffman  and  Ritchie  (1997) 
proposed  that  the  learners  in  the  classroom  become  researchers  and  work  in  small  groups  to 
analyze  problems,  determine  solutions,  and  evaluate  problems  when  utilizing  PBL.  Successful 
examples  of  using  PBL  in  computer  science  include  the  teaching  of  object-oriented  programming 
in  the  context  of  game  design  (Ryoo,  Fonseca,  &  Janzen,  2008),  learning  computer  programming 
logic  via  PBL  (Pereira,  Zebende,  &  Moret,  2010),  and  in  Anny  leader  development  using  the 
case  method  (Zyblut,  Brunner,  &  Vowels,  2007). 

Studio-Based  Learning  (SBL)  has  its  origins  in  design  fields,  such  as  architecture  and  art. 
Lackney  (1999)  provided  an  overview  of  SBL  history,  tracing  its  roots  back  to  the  late  1800s. 
SBL  centers  on  students  constructing  their  own  solutions  to  assigned  problems,  and  receiving 
critical  review  and  participating  in  discussion  with  instructors  and  others.  Hundhausen, 

Agrawal,  Fairbrother,  and  Trevisan  (2010)  identified  the  following  four  key  steps  related  to  SBL 
processes. 
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1 .  Students  are  given  meaningful  problems  for  which  there  are  multiple  solutions  and  for 
which  they  have  to  construct  solutions  individually  or  in  groups, 

2.  Students  present  their  solutions  and  justifications  to  the  entire  class, 

3.  Their  peers  critique  the  solutions  and  provide  comments,  and 

4.  Students  are  given  the  opportunity  to  respond  to  comments  and  criticisms,  and  modify 
their  solutions  appropriately. 

However,  in  the  same  article,  the  authors  found  no  differences  in  learning  outcomes 
between  an  experimental  group  receiving  SBL  and  a  group  receiving  more  traditional 
instruction.  They  did  find  that  the  SBL  group  reported  increased  self-efficacy  and  greater 
learning  from  peers,  measured  through  self  reports.  Reardon  and  Tangney  (2011)  in  a  recent 
case  study  using  college  freshmen  as  part  of  a  computer  programming  course  found  SBL 
provided  support  for  the  motivation  and  programming  achievement  of  the  students,  though  no 
experimental  comparison  was  made. 

Certainly  both  PBL  and  SBL  strategies  can  be  found  in  AIT  and  therefore,  the  findings 
relevant  to  successful  applications  of  mobile  devices  using  these  methods  have  direct 
implications  for  AIT.  Perhaps  more  relevant  to  medical,  law,  and  other  “case”  based  tasks,  these 
methods  may  still  have  relevance  for  student  collaboration  and  communities  of  practice  within 
MOSs  or  occupational  fields.  In  those  tasks  and  MOSs  where  cases  or  problems  serve  as  the 
primary  stimuli  for  application  of  learning,  the  use  of  devices  to  implement  PBL  and  SBL 
practices  are  more  likely  to  be  effective  and  prevalent.  However,  it  is  likely  that  nearly  all  MOSs 
have  some  potential  for  case-  and  problem-based  learning  applications. 

Instructional  scaffolding.  Originating  from  cognitive  learning  theory,  scaffolding  in  the 
context  of  instruction  refers  to  instructors,  peers,  or  others  providing  support  and  structure  for 
learning.  Scaffolding  is  often  used  in  order  to  support  problem-based  learning  (PBL). 

Scaffolding  may  occur  in  fonnal  learning  settings  or  on  the  job,  through  apprenticeship  (Nielsen, 
2008),  mentoring,  and  job  shadowing,  among  other  means.  The  practice  of  scaffolding  is  related 
to  Vygotsky’s  ‘Zone  of  Proximal  Development’  which  described  the  developmental  gap  that 
exists  between  independent  problem  solving  and  that  which  could  be  achieved  through  expert 
guided  instruction  (Vygotsky,  1978). 

Salmon  (2000)  developed  a  five  stage  model  of  interactivity  and  learning  in  online 
learning  used  by  Lim  and  Siraj  (2011)  as  a  means  to  promote  scaffolding  of  mobile  learning, 
consisting  of 

1 .  Access  and  motivation  -involves  welcoming  and  encouraging  as  well  as  providing 
technical  support  during  the  set  up,  access,  and  support  of  the  use  of  technology, 

2.  Network  socialization  -involves  familiarizing  and  providing  bridges  between  cultural, 
social,  and  learning  environments  during  the  sending  and  receiving  of  messages, 

3.  Information  exchange  -involves  facilitating  tasks,  discussion  and  supporting  the  use  of 
learning  materials  during  searching,  and  other  social  applications  of  the  devices, 

4.  Context  and  knowledge  construction  -involves  facilitating  the  learning  process  and  open 
activities  during  conferencing,  and 

5.  Development  -  involves  supporting  and  responding  when  needed. 


13 


This  model  represents  a  potentially  valuable  structure  for  developing  scaffolding 
activities  in  mobile  devices,  though  it  has  not  been  researched  extensively.  Scaffolding,  in 
general,  is  likely  a  critical  topic  for  examining  how  mobile  devices  would  be  applied  in  AIT  to 
ensure  instructors  are  monitoring,  supporting,  and  evaluating  Soldier  learning  and  aware  of  the 
potential  for  information  that  might  lead  to  “negative”  training.  Mobile  devices  offer  a 
capability  to  extend  the  reach  of  the  instructor  beyond  the  classroom  and  range  setting,  as  well  as 
the  scheduled  limits  of  training  time.  Scaffolding  practices  also  offer  a  means  through  which  to 
better  communicate  the  value  of  implementing  mobile  devices  as  a  means  to  assist,  rather  than 
replace,  the  instructor.  This  may  be  important  as  reluctance  based  on  these  probable  instructor 
concerns  could  prove  to  be  a  major  barrier  to  integrating  mobile  devices  in  AIT. 

Experiential  learning.  Experiential  learning  was  defined  by  Kolb  (1984)  as  “the  process 
whereby  knowledge  is  created  through  the  transfonnation  of  experience.  Knowledge  results 
from  the  combination  of  grasping  and  transfonning  experience”  (p.  41).  In  this  sense, 
experiential  learning  can  be  seen  as  an  extension  of  constructivism.  Kolb’s  Experiential 
Learning  Theory  proposed  that  cycles  of  action,  feedback,  reflection,  adaption,  and  abstraction 
allow  students  to  learn  by  doing  and  reflecting  upon  their  experiences.  The  distinction  between 
experiential  learning  and  more  traditional  fonns  of  learning  using  instructor  guidance  is  that 
experiential  learning  begins  with  action  or  practicing  what  is  to  be  learned  instead  of  instruction 
or  guidance.  A  corresponding  Learning  Styles  Inventory  was  developed  to  assess  orientation 
toward  experiential  learning  (Kolb,  1976).  However,  the  effects  of  learning  style  on  learning 
transfer  have  come  under  considerable  criticism  with  little  empirical  evidence  supporting 
learning  style  effects  on  learning  outcomes  (Bergsteiner,  Avery,  &  Neumann,  2010). 

Experiential  learning,  in  the  context  of  training,  would  be  most  related  to  the  concept  of 
practice,  feedback,  and  input  from  others  to  improve  performance.  These  elements  are  integral 
to  AIT.  However,  expanding  experiences  to  mobile  devices  requires  careful  attention  to  the 
fidelity  of  the  mobile  device  and  learning  experience  to  ensure  appropriate  video  and  audio 
stimuli,  equipment  responses,  and  feedback  are  provided.  There  are  limits  to  the  capability  of 
mobile  devices  to  provide  high  fidelity  training,  particularly  in  highly  technical  and  tactile 
environments.  As  simulation  capabilities  continue  to  emerge  through  mobile  devices  and 
peripherals  become  more  prevalent,  the  experiential  aspect  of  mobile  device  use  will  improve, 
but  there  will  likely  always  be  significant  limitations  to  mobile  device  capabilities  compared 
with  real  equipment  and  range  conditions.  Partially  offsetting  these  limitations  is  the  central 
advantage  of  mobile  devices  to  allow  more  students  in  different  environments  to  experience 
training,  in  whatever  form  (e.g.,  lecture,  observation,  gaming,  etc.),  in  a  more  timely,  cost 
effective  manner.  So,  while  the  experience  may  not  be  as  robust,  the  ability  to  share  the 
experience  and  deliver  it  in  any  number  of  physical  environments  is  greatly  increased. 

Situated  and  authentic  learning.  Following  from  the  constructivist  theory  of  learning, 
and  also  closely  related  to  experiential  learning,  are  situated  and  authentic  learning  practices  that 
are  based  upon  the  notion  that  learning  occurs  and  is  influenced  by  the  culture,  context,  and 
activities  in  which  the  perfonnance  takes  place  (Lave,  1988).  Lave  and  Wenger  (1991) 
described  learning  as  not  merely  the  acquisition  of  knowledge  but  instead  a  process  of  social 
participation.  Situated  learning  then  requires  that  “knowledge  be  presented  in  authentic  contexts 
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(settings  and  application  that  would  normally  involve  that  knowledge)  and  learners  to  participate 
within  a  community  of  practice”  (Naismith  et  al.  2004,  p.  13). 

Unfortunately,  there  is  not  much  empirical  literature  related  to  the  effectiveness  of 
situated  and  authentic  learning.  However,  the  literature  of  simulation  fidelity  (e.g,  Allen,  Hays, 
&  Buffardi,  1986;  Issenberg,  McGaghie,  Petrusa,  Gordon,  &  Scalese,  2005)  could  be  extended  to 
provide  support  for  the  notion  that  the  greater  the  similarity  between  the  training  context  and  the 
perfonnance  environment,  the  greater  the  transfer  of  that  training. 

As  applied  to  technical  training,  the  implications  of  situated  and  authentic  learning 
support  prevailing  institutional  and  unit  training  strategies,  particularly  training  using  actual 
equipment  under  field  conditions.  In  terms  of  their  applicability  to  AIT,  the  use  of  ranges,  actual 
equipment,  and  conditions  which  simulate  or  replicate  real  world  conditions  would  be  viewed  as 
more  effective  than  classroom,  instructor  led,  or  even  remote  environments  that  cannot  fully 
replicate  on-the-job  conditions.  Many  of  the  issues  identified  with  experiential  learning  with 
respect  to  mobile  devices  are  relevant  to  situational  and  authentic  learning  practices. 

Collaborative  and  cooperative  learning.  Collaborative  and  cooperative  learning  (the 
tenns  are  used  interchangeably)  involve  interactions  between  learners  to  improve  the  learning 
experience.  Cooperative  learning  has  been  defined  as  working  together  with  another  person  or 
group  to  accomplish  shared  goals  (Lefrancois,  1999,  p.  539).  Johnson  and  Johnson  (1994) 
described  three  types  of  learning:  individual,  competitive,  and  cooperative. 

A  number  of  research  efforts  have  demonstrated  that  cooperative  learning  increases 
learner’s  motivation  when  compared  with  traditional,  instructor  led  approaches  (Johnson  & 
Johnson,  1989;  Slavin,  1995;  Sharan  &  Shaulov,  1990).  In  addition,  collaborative  activities  have 
been  found  to  significantly  improve  the  effectiveness  of  a  learning  experience,  increasing  both 
individual  and  group  performance  (Webb  &  Palincsar,  1996).  Looi,  Chen,  and  Ng  (2010)  found 
that  use  of  a  collaborative  software  technology  resulted  in  higher  assessment  scores,  more 
opportunities  to  participate  in  class  discussion,  and  exposure  to  diversified  ideas  for  students  in 
primary  school  science  classes.  Austin,  Smyth,  Rickard,  Quirk-Bolt,  and  Metcalfe  (2010) 
provided  an  overview  of  the  empirical  findings  in  support  of  the  value  of  collaborative  learning 
including  deeper  understanding  and  knowledge  of  others  (Bonk  &  King,  1998;  Kasper,  2000), 
increased  focus  on  problem  solving  skills  as  a  result  of  engaging  with  others  (Damon,  1984; 
Johnson  &  Johnson,  1988),  and  long-term  retention  of  learned  material  (Garrison,  Anderson,  & 
Archer,  2001;  Johnson  &  Johnson,  1999). 

Mobile  devices  offer  significant  promise  related  to  improving  the  methods,  availability, 
and  outcomes  associated  with  collaborative  and  cooperative  learning.  Mobile  devices  have 
already  demonstrated  their  ability  to  facilitate  social  networking  for  collaboration  and  that  can  be 
taken  advantage  of  in  various  training  settings. 

Computer-assisted  and  computer  based  learning  and  instruction.  For  decades,  the 
personal  computer  has  dominated  our  work,  information  processing  and  access,  and  learning 
domains.  The  use  of  the  computer  to  improve  human  performance  and  learning  occurs  primarily 
through  the  ability  of  various  devices  to  provide: 
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1 .  Information  access  and  networking, 

2.  Self  directed  and  self-controlled  sequencing  (linking), 

3.  Visual  and  auditory  stimulation  and  simulation, 

4.  Simulated  tool  control  and  interactivity,  and 

5.  Performance  enhancement,  feedback,  and  support  applications. 

For  MOSs  that  rely  on  information  and  processing,  the  computer  is  indispensable  and 
users  must  understand  their  computer’s  capabilities  and  functions  to  carry  out  their 
responsibilities.  Learning  with  computers  is  an  expected  and  required  part  of  their  training  and 
daily  routine. 

For  those  tasks  that  require  psychomotor  skills,  computer  based  learning  has  increased 
access  to  and  practice  with  the  functional  tools  while  reducing  costs,  providing  safe  practice 
environments,  and  giving  timely  feedback  that  is  often  not  possible  under  operational  conditions. 
The  contribution  of  handheld  devices  to  psychomotor  skill  development  has  been  demonstrated 
with  various  recent  generation  gaming  systems.  Fundamental  movements  related  to  a  wide 
variety  of  sports  such  as  golf,  bowling,  table  tennis,  etc.  are  encouraging  users  to  mimic  the 
actual  motions  used  during  these  sports.  Marksmanship  trainers  using  computer  driven  video 
screens,  modified  weapons,  and  weapon  effects  can  streamline  their  skill  development  in 
barracks,  aboard  ships,  and  other  locations.  However,  it  should  be  noted  that  these  fundamental 
movements  may  actually  encourage  negative  behaviors,  and  therefore  it  is  critical  to  determine 
the  tool-system  interface  fidelity  and  integrated  assessments  when  assessing  the  value  of  such  a 
system  in  transfer  of  skills. 

The  capabilities  of  mobile  computing  devices  to  extend  the  effects  of  computer-based  and 
assisted  instruction  are  emerging  but  largely  unexplored.  Many  social,  game,  and  other 
applications  contribute  to  navigation  and  enjoyment,  but  learning-specific  applications  are 
primarily  focused  on  traditional  delivery  fonnats  (e.g.,  books,  photos,  videos)  and  testing 
practices.  This  review  has  identified  a  variety  of  learning  theories  and  practices  related  to 
detennining  how,  when,  where,  and  perhaps  most  importantly  why,  these  devices  can  improve 
existing  training  practices  to  address  real  world  performance  issues.  The  following  section 
addresses  representative  lessons  learned  organized  by  device  characteristics,  rather  than  learning 
theory  constructs. 

Mobile  Training  Device  Lessons  Learned 

The  learning  theories  reviewed  previously  are  generally  applicable  to  most  training 
contexts,  including  AIT.  There  is  another  line  of  research  that  is  applicable  to  using  mobile 
devices  in  AIT.  While  the  learning  theories  clearly  focus  on  instructional  methods,  the  other  line 
of  research  focuses  on  the  mobile  learning  device  itself.  This  line  of  literature  is  mostly 
theoretical;  however,  recent  research  studies  have  implemented  mobile  device  learning  and 
provided  some  informative  observations.  These  observations  serve  as  lessons  learned  for 
implementing  mobile  learning.  Most  of  the  literature  discussed  in  this  section  considers  non¬ 
military  training  contexts  and  recommendations  from  mobile  learning  theorists  from  academia. 
Insights  regarding  military  training  contexts  were  gained  from  discussions  with  key  personnel 
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overseeing  the  Army’s  Connecting  Soldiers  to  Digital  Applications  (CSDA),  pilot  tests,  which 
explored  the  usability  and  utility  of  mobile  devices  specifically  in  an  Army  training  context. 

The  literature  on  lessons  learned  regarding  mobile  learning  can  be  sorted  into  two 
domains:  usability  and  utility.  Usability  topics  include  basic  device  functionality  in  the  training 
context  and  learner  training  requirements.  Utility  concerns  the  usefulness  of  mobile  training 
devices  for  training.  In  this  section,  utility  is  discussed  for  both  learners  and  instructors.  Also 
mobile  training  effectiveness  literature  is  reviewed  to  identify  the  training  outcomes  that  are 
affected  by  mobile  devices. 

Usability.  Usability  literature  directly  addresses  many  important  ease-of-use  issues 
related  to  implementing  mobile  learning.  Low  and  O’Connell  (2006)  suggested  that  as  usability 
was  a  catch  word  concerning  the  design  of  computer  interfaces,  it  should  also  be  considered 
along  with  the  concept  of  user-centric  learning  in  developing  mobile  resources.  Usability  issues 
need  to  be  evaluated  within  the  training  context  for  each  block  of  instruction  in  order  to 
detennine  if  a  mobile  learning  approach  should  be  implemented.  Mobile  learning  is  meant  to 
facilitate  the  learning  process  (Jeng,  2010).  Usability  does,  of  course,  have  bearing  on  the 
devices’  utility:  if  the  usability  is  so  poor  that  it  hinders  learning,  then  there  is  limited  utility.  If 
the  use  of  technology  does  not  add  value  to  the  learning  process,  then  it  is  unwarranted.  To  aid 
in  the  evaluation  of  usability  the  following  topics  are  discussed:  usage  guidelines,  security, 
device  requirements,  cost,  and  mobile  device  limitations. 

Usage  guidelines.  Usage  guidelines  are  required  to  identify  when  users  will  be  allowed 
to  possess  mobile  devices  and  what  user  pennissions  they  will  have.  Usage  guidelines  suggest 
how  mobile  learning  devices  will  be  used  formally  and  informally  by  learners  in  AIT.  This 
section  considers  guidelines  to  prevent  misuse  and  provide  infonnation  security. 

The  Anny  Learning  Model  (2011)  identified  the  capability  of  mobile  learning  devices  to 
be  a  disruptive  technology.  The  disruptive  uses  and  potential  for  distraction  need  to  be 
considered  in  creating  instructional  guidelines  and  policy.  The  CSDA  pilot  testing  provided 
examples  of  how  phones  can  become  distractors  without  effective  guidelines  and  policies 
concerning  usage.  In  the  fonnal  instructional  environment,  some  Soldiers’  attention  was 
diverted  from  instructors  (e.g.,  personal  e-mail,  playing  games).  These  distractions  take  away 
from  the  current  effectiveness  of  the  AIT  training  context. 

CSDA  pilot  testing  noted  that  distraction  problems  were  most  likely  to  occur  in  Soldiers 
who  are  inexperienced  and  still  coming  to  understand  Anny  behavior  and  acceptable  Anny 
equipment  use.  Distractions  can  be  minimized  by  explaining  the  proper  use  of  the  phone  during 
training  and  clarifying  user  restrictions. 

The  challenge  to  establishing  guidelines  for  effective  use  is  that  innovative  and  creative 
uses  for  mobile  devices  should  not  be  stifled.  Sharpies  (2009)  discussed  the  unpredictable  nature 
of  mobile  device  usage  and  identified  three  types  of  unpredictability  related  to  mobile  devices. 
First,  the  context  of  mobile  device  use  is  unpredictable.  Context  of  use  refers  to  the  specific 
circumstances  that  explain  the  environment  and  how  users  will  be  prompted  to  use  mobile 
devices  (e.g.,  for  an  assignment,  or  informally  based  on  their  interests).  This  unpredictability  can 
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remove  the  distinction  between  fonnal  and  informal  learning  environments  and  introduce 
uncertainty  as  to  how  mobile  devices  will  be  used.  Second,  the  learning  process  is  unpredictable. 
The  learning  process  refers  to  the  types  of  activities  that  the  learner  will  engage  in  (e.g.,  reading, 
watching,  collaborative  exercises).  Learners  can  be  guided  through  training  by  instructors,  but 
the  individual  Soldier’s  learning  process  and  use  of  mobile  devices  in  training  will  differ.  Lastly, 
the  mode  of  use  is  unpredictable.  The  mode  of  use  refers  to  how  learners  use  mobile  device 
capabilities  (e.g.,  the  Internet,  camera,  etc.).  The  full  extent  of  how  technology  and  its 
capabilities  can  be  leveraged  and  exploited  is  unknown  until  it  is  placed  in  the  hands  of  users. 
These  examples  of  unpredictability  depict  the  challenges  associated  with  creating  adequate 
guidelines  while  not  deterring  innovation  and  creative  application. 

Security.  The  Army  Learning  Model  (2011)  identified  the  need  to  quickly  resolve  and 
implement  security  safeguards  concerning  mobile  devices  to  exploit  the  usefulness  of  mobile 
devices  for  learning.  Security  safeguards  must  address  usage  restrictions.  Mobile  learning 
devices  link  learners  to  the  outside  world  (Naismith  et  ah,  2004).  While  this  may  become  a 
distraction  in  formal  training  contexts,  it  can  be  a  benefit  in  informal  learning.  However, 
informal  training  contexts,  such  as  the  barracks,  are  less  easily  monitored  and  thereby  more 
susceptible  to  misuse.  Naismith  et  al.  noted  the  importance  of  providing  security  and  privacy  for 
learners. 

Given  the  communication  restrictions  that  are  placed  on  Soldiers  during  AIT,  training 
designers  must  decide  what  level  of  mobile  device  functionality  and  connectivity  is  appropriate 
for  Soldiers.  TRADOC  350-6  Training:  Enlisted  Initial  Entry  Training  (IET)  Policies  and 
Administration  states  that  cell  phones,  pagers,  and  electronic  devices  are  subject  to  control 
during  Phase  IV  and  V  of  AIT.  During  these  phases,  “Privileges  are  earned,  and  are  granted, 
reduced  or  modified  at  the  commander’s  discretion  based  on  Soldier  performance  and  discipline 
(TRADOC  350-6,  p.  20).”  Such  privileges  often  include  use  of  a  cell  phone  or  computer.  In  the 
CSDA  pilot  testing,  some  Soldiers  in  AIT  were  disciplined  for  removing  the  memory  card  and 
changing  user  restrictions.  Security  policies  and  usage  restrictions  should  be  accounted  for  in 
mobile  training  device  programming  and  application  design  and  Soldiers  should  be  made  aware 
of  policies,  restrictions,  and  consequences  for  tampering  with  a  mobile  device’s  user  restrictions 
or  security  systems. 

Training  needs.  Despite  some  Soldiers’  personal  experience  with  mobile  devices,  most 
Soldiers  still  require  training  (at  least  on  their  use  as  Anny  equipment  and  functionality),  as  they 
would  with  any  new  type  of  equipment;  Army  Learning  Model  2011).  CSDA  pilot  tests  found 
that  after  Soldiers  were  trained  on  the  devices,  they  said  that  they  were  more  capable  of  using  the 
phones.  These  findings  suggest  that  Soldiers  can  benefit  from  training  on  how  to  use  their 
mobile  devices. 

Soldier’s  technological  capacity  and  previous  experience  with  mobile  devices  can  be 
misleading.  Each  Soldier’s  experiences  will  vary.  This  combined  with  the  unpredictability  of 
how  Soldiers  will  use  the  devices  suggests  Soldiers  should  be  trained  in  the  functions  of  the 
device  as  well  as  how  to  use  the  specific  applications  that  will  be  used  in  training.  This  type  of 
training  will  ensure  the  integration  of  mobile  devices  into  the  learning  process.  Additionally, 
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training  should  be  provided  to  identify  Anny  guidelines  and  policies  concerning  mobile  devices, 
as  would  be  customary  for  any  other  equipment. 

Similar  to  Soldiers,  instructor’s  level  of  mobile  device  experience  will  vary.  Instructor 
experience  level  can  impact  how  instructors  are  expected  to  use  devices  in  teaching.  Instructors 
need  to  have  a  mastery  of  mobile  device  instruction.  Just  as  instructors  would  be  seen  as  an 
expert  on  any  other  piece  of  equipment  that  they  use  to  train,  so  too  will  they  need  to  have 
command  of  mobile  devices.  An  important  difference  between  mobile  devices  and  other  Anny 
equipment  is  that,  in  some  instances,  students  could  have  more  experience  with  mobile  devices 
than  instructors.  Instructors  may  require  training  on  mobile  device  functions  and  the  utility  of 
devices  for  their  courses. 

Mobile  device  functions.  Research  has  identified  several  variables  to  consider  for 
choosing  a  mobile  device.  Device  functions  have  improved  and  new  hardware  has  led  to  new 
instructional  and  software  application  capabilities.  Research,  including  the  CSDA  pilot  tests  has 
also  examined  user  preferences  regarding  device  functions  and  capabilities.  A  mobile  learning 
device’s  instructional  and  software  application  capabilities  are  dependent  upon  device 
functionality  (e.g.,  hardware  and  the  supporting  infrastructure).  In  some  instances  shortcomings 
related  to  device  hardware  and  infrastructure  can  be  overcome  through  effective  instructional 
design.  Low  and  O’Connell  (2006)  suggested  that  individuals  responsible  for  designing  and 
implementing  mobile  learning  need  to  become  educated  and  aware  of  the  multiple  platfonns  and 
formats  available  to  understand  the  quality  and  requirements  that  are  most  suited  for  their  usage. 

Device  functions  need  to  be  taken  into  consideration  prior  to  selecting  a  device,  and  once 
a  device  is  selected  a  comprehensive  list  of  capabilities  needs  to  be  identified.  A  list  of 
capabilities  should  guide  design.  Desired  capabilities  could  be  used  to  identify  how  technology 
could  be  leveraged  to  achieve  learning  objectives  and  improve  instructional  techniques.  Learner 
activities  and  other  instructional  capabilities  are  dependent  upon  mobile  device  functions.  Jeng 
(2010)  and  Klopfer,  Squire,  and  Jenkins  (2002)  described  two  methods  for  identifying  device 
capabilities  and  providing  objectives  or  examples  as  to  how  the  device  capabilities  can  be  used. 
Jeng  identified  capabilities  related  to  basic  functions  of  mobile  devices  (e.g.,  video  recording) 
and  Klopfer  et  al.  identified  capabilities  related  to  how  the  user  can  use  mobile  devices  (e.g., 
portability). 

Some  of  the  basic  functions  available  on  mobile  devices  are  network  connectivity,  picture 
taking,  video  recording,  a  global  positioning  system  (GPS)  receiver,  and  radio  frequency 
identification  (RFID)  readers  (Jeng,  2010).  Each  of  these  functions  could  be  leveraged  during 
training.  More  functions  on  a  device  means  more  capabilities  for  training  designers,  instructors, 
and  learners.  A  mobile  device  should  be  selected  based  on  its  functional  capabilities.  Following 
the  selection  of  a  device,  functions  should  be  listed  in  detail  and  compared  to  instructional 
practices  to  benefit  training  designers. 

Klopfer  et  al.  (2002)  created  a  taxonomy  for  understanding  the  capabilities  of  mobile 
devices.  The  factors  in  the  taxonomy  are  portability,  social  interactivity,  context  sensitivity, 
connectivity,  and  individual  differences  (e.g.,  age,  familiarity  with  mobile  devices,  etc.). 
Similarly,  Tucker  (2010)  identified  several  capabilities  that  are  common  to  mobile  devices: 
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connectability,  portability/wearability,  instant  accessibility,  flexibility,  economic  viability,  social 
interactivity,  context  sensitivity,  and  individuality.  The  literature  on  learning  theories  and 
instructional  techniques  is  most  relevant  to  the  social  interactivity,  context  sensitivity,  and 
individual  differences  factors.  Those  factors  are  very  interrelated  to  the  training  context  and 
instructional  strategies.  This  section  concerns  the  factors  that  are  most  specifically  related  to 
using  mobile  devices,  portability,  durability,  and  connectivity. 

Portability.  Portability  refers  to  the  device  size  and  ability  for  devices  to  be  easily  carried 
by  users.  The  CSDA  pilot  tests  identified  portability  and  screen  size  as  limitations.  These 
variables  often  opposed  one  another  (e.g.,  as  screen  size  increases  portability  decreases). 
Similarly,  the  recommendations  were  presented  as  trade-offs.  Soldiers  wanted  a  small  device  to 
ensure  portability,  but  noted  that  device  features  and  application  usage  is  degraded  if  the  device 
is  too  small.  Similarly,  Suki  and  Suki  (2011)  found  that  learners  objected  to  the  screen  size, 
specifically  because  the  training  they  received  prior  to  the  learning  device  was  delivered  on  a 
larger,  more  user  friendly,  screen. 

Low  and  O’Connell  (2006)  suggested  two  guidelines  that  could  help  address  the  issue  of 
screen  size.  First,  choose  fonts  and  graphics  that  are  easy  to  view  and  second,  carefully  create 
pages  to  prevent  scrolling  in  two  dimensions.  Additionally,  designers  should  eliminate 
unnecessary  code,  content,  and  operations;  use  space  efficient  fonnats  and  appealing  color 
palettes  for  creating  media;  and  divide  content  into  small  objects  so  users  can  download  what 
they  need.  These  guidelines  are  an  example  of  design  features  that  can  be  used  to  address 
limitations  of  screen  size. 

Durability.  Durability  refers  to  the  ability  of  a  device  to  be  used  and  not  get  damaged  in 
harsh  environments.  Durability  is  related  to  a  device’s  portability  as  fragile  devices  become 
more  susceptible  to  damage  and  therefore  less  useful.  Soldiers  suggested  that  a  lack  of  durability 
in  a  mobile  learning  device  is  an  impediment  to  its  use.  This  limitation  can  lead  to  infrequent 
use  in  field  environments,  especially  when  field  training  is  unpredictable  (e.g.,  inclement 
weather,  physically  rigorous  training  conditions).  It  is  possible  to  increase  the  durability  of 
devices  by  adding  additional  carrying  cases  to  the  devices.  However,  this  equipment  is  an 
additional  cost  and  does  not  completely  eliminate  durability  issues. 

Connectivity.  Mobile  device  connectivity  capabilities  include  networking  mobile 
devices,  connecting  users  to  a  central  network,  and  linking  users  to  the  internet  (Klopfer  et  ah, 
2002;  Naismith  et  ah,  2004).  Connectivity  capabilities  expand  training  options  and  meet  the 
desires  of  trainees.  More  access  to  the  internet  and  improved  internet  connectivity  was  the  most 
frequent  response  from  Soldiers  concerning  additional/desired  features  for  mobile  devices. 
Soldiers  wanted  access  to  e-mail,  AKO,  Army  Times,  and  social  networking.  Connectivity 
capabilities  require  security  considerations  and  usage  guidelines.  Once  usage  policies  have  been 
established,  connectivity  can  be  leveraged  as  a  resource  to  meeting  learning  objectives  and 
enhancing  instructional  techniques. 

Utility.  Utility  concerns  the  effectiveness  of  using  mobile  devices  for  training.  This 
section  will  discuss  research  findings  that  describe  how  mobile  devices  are  useful  to  learners  and 
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instructors.  Further,  the  utility  of  mobile  learning  is  reviewed  by  examining  the  effect  of  mobile 
learning  on  training  outcomes. 

Learners.  Low  and  O’Connell  (2006)  created  a  taxonomy  to  identify  mobile  learning 
activity  from  a  learner-centric  perspective.  Using  a  mobile  device,  a  learner  can  record, 
reinterpret,  recall,  and  relate.  Low  and  O’Connell  suggested  the  recording  activity  is  based  on  a 
constructivist  type  of  learning.  Learners  gather  and  build  information  during  training.  In  CSDA 
pilot  tests,  Soldiers  identified  the  ability  to  take  pictures  and  video  as  advantages  to  mobile 
learning  in  Army  training.  In  this  effort,  mobile  devices  were  effective  in  tactical  training 
contexts  for  gathering,  sharing,  and  storing  information. 

The  reinterpret  type  of  mobile  learning  activity  consists  of  the  learner  analyzing  and 
transforming  existing  knowledge  and  resources  (Low  and  O’Connell,  2006).  The 
reinterpretation  of  data  and  resources  provides  the  user  an  additional  tool  for  learning  to  capture 
and  evaluate  what  is  learned  in  fonnal  training.  For  example,  a  Soldier  could  select  relevant 
information  from  courses,  training  materials,  and  other  materials  to  enter  into  a  training  journal, 
to  create  a  timeline  of  their  learning,  or  help  them  remember  the  most  important  things  they 
learned.  Reinterpretation  relies  on  the  processing  abilities  of  the  mobile  device  to  store 
information  and  the  learner’s  ability  to  analyze  existing  sources  of  infonnation. 

The  recall  activity  consists  of  learners  using  the  mobile  device  to  review  existing 
information  and  resources  (Low  &  O’Connell,  2006).  Low  and  O’Connell  suggested  that  recall 
activity  is  based  on  connectivist  theories  if  training  is  learner-centric.  Learners  recall 
information  as  they  explore  their  environment  and  make  connections  between  what  they  know 
and  infonnation  from  resources  to  expand  their  network  of  knowledge.  Instructivist  theories  are 
applicable  to  training  with  mobile  devices  if  recall  is  teacher-centered.  Instructors  would  guide 
and  facilitate  the  acquisition  of  knowledge  by  identifying  resources  for  recall.  Soldiers  in  AIT 
could  access  existing  resources  as  they  see  fit  or  benefit  from  existing  resources  with  instruction 
and  facilitation  from  an  instructor.  In  CSDA  pilot  tests,  Soldiers  in  AIT  most  frequently  used 
mobile  devices  during  training  to  prepare  for  classes  and  exams  and  refresh  learning.  Further, 
when  Soldiers  were  asked  to  identify  the  advantages  to  using  mobile  devices,  access  to 
information  and  studying  were  among  the  top  responses.  Soldiers  identified  videos  and 
interactive  applications  as  the  most  desired  mobile  learning  methods  of  instruction.  Refresher 
videos  and  applications  can  reinforce  material  learned  during  formal  training. 

The  relate  activity  considers  the  ability  for  mobile  devices  to  link  learners  to  one  another 
to  create  a  social  context  and  network  of  knowledge  (Low  &  O’Connell,  2006).  Learners  can 
network  and  leam  with  (and  from)  peers.  Collaborative  learning  theories  can  be  used  to 
maximize  the  quality  of  interactions  among  peers.  For  example,  a  collaborative  approach  to 
learning  could  link  Soldiers  together,  opening  each  individual  as  a  resource  to  others.  A  linkage 
among  learners  would  allow  for  cooperation  and  collaboration  in  group  assignments.  In  CSDA 
pilot  tests,  Soldiers  who  had  connectivity  were  able  to  share  video  and  photographs  to  support 
their  mission.  Lastly,  relate  includes  networks  created  between  learners  and  instructors. 
Instructors  could  use  mobile  devices  to  reach  beyond  the  classroom  and  provide  additional 
support  to  students. 
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Instructors.  Results  from  CSDA  pilot  tests  suggested  that  instructors  disliked  mobile 
devices,  in  some  cases  because  the  content  on  the  mobile  devices  did  not  parallel  what  they  had 
been  teaching  throughout  the  rest  of  the  course.  This  is  contrary  to  Anny  guidelines  for  an 
integrated  training  environment  (Anny  Training  Concept,  2011).  Holden  and  Westfall  (2010) 
identified  the  ability  to  update  content  rapidly  as  an  important  aspect  of  a  mobile  device.  The 
subject  matter  that  is  being  trained  must  be  examined  to  identify  how  likely  it  will  be  to  change. 
Instructors  and  designers  need  to  identify  aspects  of  their  courses  that  will  require  considerable 
content  updates.  If  content  changes  too  frequently  then  it  would  require  frequent  updating 
making  it  less  suitable  for  mobile  devices  due  to  the  increased  cost  of  updates.  Instructors  could 
be  given  the  ability  to  make  changes  directly  to  content,  but  they  would  need  to  be  provided 
training  and  technical  support  to  ensure  that  they  are  able  to  improve  and  adapt  their  course 
content  (Naismith,  Lonsdale,  Vavoula,  &  Sharpies,  2004). 

The  integration  of  mobile  devices  into  AIT  is  dependent  upon  the  instructor’s  level  of 
involvement.  To  ensure  seamless  integration,  instructors  should  be  consulted  in  the  development 
of  training  applications.  Including  instructors  in  the  design  process  improves  the  quality  of 
training,  ensuring  that  content  is  accurate.  Further,  instructor  involvement  in  design  encourages 
instructors  to  buy  into  the  inclusion  of  mobile  devices  in  training. 

Criteria  for  evaluating  mobile  learning  outcomes.  There  is  a  lack  of  experimental 
evidence  regarding  the  effectiveness  of  mobile  devices  for  learning  (Tucker,  2010).  In  the 
absence  of  studies  that  explore  a  causal  relationship  between  mobile  learning  and  the 
effectiveness  of  training  outcomes,  there  has  been  survey  and  interview  research  conducted  using 
learner  and  instructor  samples  that  examines  perceptions  of  effectiveness.  This  research 
suggested  that  learners’  performance  can  improve  using  mobile  devices  and  that  other  indirect 
perfonnance  factors  such  as  motivation  and  communication  were  also  improved. 

The  data  collected  from  surveys  of  Soldiers  who  participated  in  CSDA  pilot  tests 
suggested  that  Soldiers  believe  there  are  many  positive  benefits  from  using  mobile  devices. 
Soldiers  identified  several  impacts  directly  related  to  learning,  such  as  understanding  the  course, 
retention  of  what  they  learned,  and  awareness  of  lesson  content.  Additionally,  Soldiers  said  that 
mobile  devices  increased  time  spent  studying,  increased  motivation  to  learn,  and  improved 
interactions  with  classmates  and  instructors  (e.g.,  easier  communications  via  text).  Regarding 
perfonnance  improvements,  mobile  devices  were  perceived  to  be  more  beneficial  to  Soldiers  as 
the  pace  of  their  training  mission  decreased.  It  was  not  possible  to  collect  objective  data  on  their 
direct  performance  impact  during  this  research  effort. 

Mobile  learning  has  been  shown  to  increase  learners’  motivation.  Students  reported  that 
mobile  learning  improved  their  interactions  and  increased  their  motivation  to  learn  (Motiwalla, 
2007;  Zurita  &  Nussbaum,  2007).  Students’  perceptions  of  their  increased  motivation  to  learn 
suggest  that  mobile  devices  can  be  a  tool  to  improve  learning.  Motivational  theories  in  mobile 
learning  literature  identified  several  reasons  why  motivation  is  improved:  control  over  goals, 
ownership  of  learning,  learning  with  mobile  devices  is  more  fun,  improved  communication,  and 
learning  in  context  and  continuity  between  contexts  (Jones,  Issroff,  &  Scanlon,  2007;  Sharpies, 
2009).  Notably,  these  reasons  overlap  with  learning  theories  and  instructional  techniques,  (e.g., 
self-directed  learning,  goal  setting). 


22 


Summary 

Factors  that  require  consideration  when  implementing  mobile  devices  into  AIT.  Our 

review  of  the  published  literature  has  identified  various  factors  that  need  to  be  taken  into 
consideration  when  implementing  mobile  devices  into  AIT.  By  proactively  considering  these 
various  issues  decision  makers  will  be  more  effective  at  designing  and  implementing  training 
using  mobile  devices.  Based  upon  the  literature,  our  synthesis  of  the  existing  theory  and  practice 
related  to  the  use  of  mobile  devices  in  training  identified  the  following  factors:  training  context, 
user,  and  device.  These  factors  and  their  respective  considerations,  recommendations,  and 
guidelines  are  presented  below.  Future  data  collections  will  add  to  and  elaborate  on  this  list  of 
factors,  considerations,  and  recommendations. 

Training  context  considerations. 

1 .  Content/Training  objectives  -  The  focus  of  determining  the  value  of  supplementing, 
replacing,  expanding,  and  integrating  AIT  content  using  mobile  devices  should  start  with 
an  examination  of  the  objectives  of  the  training.  Content  issues  include  the  following 
items: 

a.  Basic  knowledge,  skills,  and  behaviors  individuals  need  to  become  Soldiers  (FM 
7-0).  Skill  acquisition  and  technical  training  is  especially  important  in  AIT  due  to 
focus  on  MOS-specific  training. 

b.  Training  requirements  on  specific  equipment  and  systems. 

c.  Fonnat  of  content  -  text,  audio,  video,  interactive  media,  etc. 

d.  Proficiency  standards  and  criteria  for  evaluating  MOS  specific  training:  mobile 
devices  can  improve  performance  and  indirect  performance  factors  such  as 
motivation  and  communication. 

e.  Knowledge,  Skill,  and  Psychomotor  aspects  of  each  MOS. 

f.  The  consistency  of  course  content. 

2.  Learning  environment  -  The  learning  environment,  which  has  expanded  beyond  the 
classroom  and  field  to  include  online,  mobile,  and  application  driven  modes,  must  be 
evaluated  for  fidelity,  cost,  and  transfer  aspects.  Learning  environment  issues  include  the 
following  items. 

a.  The  mix  of  instructor  led,  self  directed,  and  mobile  training, 

b.  The  amount  of  collaboration  within  the  environment, 

c.  The  nature  of  the  learning  opportunities  and  the  focus  on  knowledge,  skill 
acquisition,  practice,  and  feedback. 

User  considerations. 

3.  Instructional  Theories  -  Instructional  theories  can  be  leveraged  in  the  design  of  mobile 
training.  The  instructional  theories  include  the  following  items. 
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a.  Behaviorism  and  Operant  Conditioning  -  How  can  mobile  devices  be  used  to 
correctly  portray  the  conditions,  stimulate  responses,  and  provide  feedback  to  the 
learner? 

b.  Cognitivism  -  How  can  mobile  devices  be  used  to  reduce  the  mental  load  on 
students  and  assist  in  organizing  the  knowledge  and  skills  to  be  learned? 

c.  Constructivism  -  How  can  mobile  devices  be  used  to  allow  the  learner  to  discover 
their  own  “truth?” 

d.  Andragogy  -  How  could  learners  use  mobile  devices  to  allow  them  autonomy  and 
the  ability  to  direct  their  own  learning  activities? 

4.  Instructional  practices  can  be  leveraged  using  mobile  devices.  Training  designers  and 
instructors  can  use  practices  in  the  design  and  implementation  of  training  using  mobile 
devices.  The  applicability  of  instructional  practices  should  be  considered.  The 
instructional  practices  include  the  following  items. 

a.  Self  Directed  Learning  (SDL)  and  Self  Regulated  Learning  (SRL)  -  SDL  and 
SRL  can  be  leveraged  using  mobile  learning  devices  by  allowing  learners  to  take 
initiative  in  their  learning,  with  control  over  their  training  and  learning  on 
individually  controlled  mobile  devices.  Further  mobile  devices  can  be  used  to 
monitor  learning  progress. 

b.  Goal  Setting  -  Mobile  devices  can  be  used  to  support  goal  setting  and  serve  as  a 
tool  for  managing  and  tracking  goals.  Also,  mobile  devices  can  be  used  to 
identify  resources  and  information  to  achieve  goals. 

c.  Problem  Based  and  Studio  Based  Learning  -  Instructors  facilitate  the  learning 
process  using  mobile  devices  through  open-ended  questioning  that  encourages 
learners  to  identify  solutions  using  mobile  device  references  and  peer 
collaboration.  Game-based  apps  can  also  utilize  problem-based  learning  approach 
for  training. 

d.  Instructional  Scaffolding  -  Mobile  devices  allow  instructors  and  peers  to  provide 
support  and  structure  for  learning. 

e.  Experiential  Learning  -  Training  using  mobile  devices  should  incorporate 
practice,  feedback,  and  input  from  others  to  improve  performance. 

f.  Collaborative  and  Cooperative  Learning  -  Mobile  devices  facilitate 
communication  and  social  networking  allowing  students  to  interact  in  a  number  of 
different  ways  with  a  large  community  of  individuals  and  groups. 

g.  Situated  and  Authentic  Learning  -  Mobile  devices  should  be  used  in  authentic 
environments  (e.g.  the  use  of  ranges,  actual  equipment,  and  conditions).  Real 
field  or  office  settings  should  be  training  contexts  considered  for  the  development 
of  mobile  training.  For  instance,  mobile  device  applications  could  be  used  during 
range  training  when  there  is  a  limited  availability  of  training  resources  for  learners 
to  get  hands  on  training. 

5.  Instructor  -  The  instructor  serves  as  the  mechanism  for  controlling  student  exposure, 
experience,  and  feedback  during  training.  Instructor  issues  include  the  following  items. 

a.  Background  and  qualifications, 
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b.  Knowledge  of  mobile  device  capabilities, 

c.  Ability  to  construct  training  support  system,  and 

d.  Instructors  can  be  resistant  to  the  use  of  mobile  devices.  They  should  be  included 
in  the  design  of  training  to  get  buy-in  and  ensure  that  mobile  applications  are 
consistent  with  their  curriculum.  Instructors  should  also  be  infonned  about 
mobile  device  capabilities  that  could  enhance  the  instructor’s  effectiveness.  For 
example,  mobile  devices  could  be  used  for  scoring  and  item  analyses  of 
embedded  knowledge  checks  or  quizzes  that  are  administered  via  the  device. 

6.  Student  -  The  student  represents  the  target  of  training  transfer  and  the  perfonnance 
mechanism.  Student  issues  include  the  following  items. 

a.  Background  and  experience, 

b.  Self  efficacy, 

c.  Readiness  for  self  directed  learning,  and 

d.  Various  other  personal  characteristics  related  to  motivation,  group  orientation, 
and  learning  orientation. 

Device  considerations. 

7.  Device  capabilities  -  Device  capabilities  describe  the  specific  functions  and  capabilities 
of  mobile  and  digital  devices.  Further,  capabilities  require  linkage  to  training  objectives. 
Device  capability  issues  include  the  following  items. 

a.  Mobile  device  selection  and  design  -  A  mobile  learning  device’s  capabilities  are 
dependent  upon  device  hardware  and  the  supporting  infrastructure.  Desired 
device  functions  and  usage  capabilities  need  to  be  taken  into  consideration  prior 
to  selecting  a  device,  and  once  a  device  is  selected  a  comprehensive  list  of 
functions  and  capabilities  needs  to  be  identified. 

b.  Basic  device  functions  -  Basic  functions  available  on  mobile  devices  need  to  be 
identified  once  a  device  is  chosen  (e.g.,  network  connectivity,  picture  taking, 
video  recording,  a  global  positioning  system  (GPS)  receiver,  and  radio  frequency 
identification  (RFID)  readers).  More  functions  on  a  device  mean  more 
capabilities  for  training  designers,  instructors,  and  learners. 

c.  Device  functions  should  be  used  to  generate  usage  capabilities  and  then 
capabilities  can  be  compared  to  learning  theories  and  instructional  practices  to 
benefit  training  designers.  This  comparison  exploits  device  functions  to  meet 
learning  objectives. 

8.  Device  Usability  and  Utility  -  Device  usability  and  utility  should  be  considered  when 
developing  training.  Usability  considers  basic  device  requirements  for  implementing 
mobile  learning  in  AIT.  Utility  considers  how  basic  device  features  can  be  used  by 
learners. 

a.  Usage  guidelines  -  Usage  guidelines  should  not  restrict  creativity  or  innovation. 
Guidelines  should  consider  use  in  fonnal  and  informal  training  contexts.  Usage 
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guidelines  should  ensure  that  devices  are  not  tampered  with  or  misused.  Usage 
guidelines  should  adhere  to  communication  restrictions  and  standards  in  AIT. 

b.  Security  -  Security  policies  and  usage  restrictions  should  be  considered  in  the 
selection  of  a  device  and  in  applications  created  for  a  device. 

c.  Training  Needs  -  Soldiers  should  be  made  aware  of  policies,  restrictions,  and 
consequences  for  tampering  with  a  mobile  device’s  user  restrictions  or  security 
systems.  The  disruptive  capabilities  of  mobile  devices  can  be  minimized  by 
explaining  the  proper  use  of  the  phone  during  training.  Additionally,  Soldiers 
should  be  trained  in  the  functions  of  the  device  as  well  as  how  to  use  applications 
that  will  be  used  in  training. 

d.  Design  recommendations  that  effect  usability  include  the  following  items. 

i.  Choose  fonts  and  graphics  that  are  easy  to  view. 

ii.  Carefully  create  pages  to  prevent  scrolling  in  two  dimensions. 

iii.  Eliminate  unnecessary  code,  content,  and  operations. 

iv.  Use  space  efficient  formats  and  color  palettes  for  creating  media. 

v.  Divide  content  into  small  objects  so  users  can  download  what  they  need. 

e.  Utility  capabilities  -  A  device’s  utility  capabilities  rely  on  training  designers’ 
ability  to  consider  and  exploit  basic  device  functions.  Utility  capabilities  need  to 
be  considered  in  the  development  of  training  to  fully  utilize  device  functions. 
These  capabilities  should  be  leveraged  for  a  purpose  (e.g.,  live  events,  online 
content,  collaboration,  assessment,  reference  materials) 

f.  Portability  -  Screen  size  limitations  and  portability  need  to  be  considered  in  the 
design  of  training  and  the  selection  of  a  training  context.  Functional  screen  size 
contributes  greatly  to  how  portable  the  device  is:  mobile  device  usage  can  be 
negatively  affected  if  devices  are  not  portable  and  if  materials  are  not  easily 
viewed. 

g.  Durability  -  Mobile  devices  need  to  be  durable  to  be  effective  in  harsh 
environments.  Devices  that  break  easily  should  require  a  protective  case. 

Training  contexts  where  devices  are  extremely  susceptible  to  damage  should  be 
minimized. 

h.  Connectivity  -  A  device’s  network  functionality  determines  user  capabilities  for 
accessing  information.  Mobile  device  connectivity  capabilities  include 
networking  mobile  devices,  connecting  users  to  a  central  network,  and  linking 
users  to  the  internet.  Designers  and  instructors  should  leverage  connectivity  to 
better  link  learners  to  training  resources.  Connectivity  should  be  used  to 
encourage  collaboration  and  cooperation  among  peers  and  with  instructors.  When 
connectivity  is  not  feasible,  designers  should  consider  a  native  app  that  still 
provides  quality  individual  training  without  requiring  access  to  a  network  or 
connections  with  other  users. 

These  factors  were  used  as  the  basis  for  developing  data  collection  instruments,  as  described 
below. 
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Method 


The  purpose  of  the  data  collection  was  to  identify  the  factors  and  considerations  that  are 
most  important  to  designing  and  implementing  mobile  learning  technologies  in  AIT.  The 
objective  of  the  data  collection  was  to  allow  the  research  team  to  further  refine  the  initial  list  of 
key  factors  and  considerations  identified  in  the  literature  review.  The  final  product  of  this 
research  was  a  practical  decision  guide  that  addressed  the  most  important  factors  and  provided 
recommendations  and  guidance  that  could  be  used  to  help  design  and  implement  digital 
applications  and  handheld  mobile  devices  in  Army  training  settings. 

Instruments 

The  instruments  used  during  this  data  collection  were  created  from  a  framework  of 
considerations  summarized  at  the  end  of  the  literature  review  (i.e.  training  context 
considerations,  user  considerations,  device  considerations).  Copies  of  these  instruments,  a 
survey,  and  a  focus  group  protocol  with  open  ended  questions,  are  provided  in  Appendix  A. 

The  paper  and  pencil  survey  consisted  of  five  sections.  In  the  first  section,  participants 
were  asked  to  provide  basic  background  infonnation,  as  well  as,  summarize  their  experience 
with  mobile  learning.  The  second  section  asked  participants  to  describe  their  experience  using 
the  instructional  practices  identified  in  the  literature  review  in  conjunction  with  mobile  devices. 
The  third  section  asked  participants  to  identify  the  potential  impact  of  using  mobile  devices  with 
various  instructional  practices  to  increase  or  enhance  learning  outcomes.  The  fourth  section 
asked  participants  to  rate  the  effectiveness  of  using  mobile  devices  in  various  training  contexts 
(e.g.,  classroom,  field/range).  In  the  fifth  section,  participants  were  asked  to  rate  the  importance 
of  various  considerations  relating  to  training  context,  users,  and  devices.  Results  from  the 
quantitative  analysis  are  provided  in  Appendix  B. 

The  focus  group  protocol  also  consisted  of  five  sections.  The  first  section  consisted  of 
questions  designed  to  learn  more  about  participants’  experience  using  mobile  devices  in  training. 
The  second  section  focused  on  the  training  context.  An  example  question  from  that  section  is, 
“What  types  of  tasks  or  training  objectives  would  be  most  appropriate  for  training  with  digital 
applications  and  mobile  technologies?”  The  third  section  concerned  user  considerations.  An 
example  question  from  that  section  is,  “What  is  the  best  way  to  ensure  instructors  buy  into  the 
use  of  digital  applications  and  mobile  technologies  in  their  training?”  The  fourth  section 
concerned  device  considerations.  An  example  question  from  that  section  is,  “Which  is  the  most 
important  device  consideration  when  integrating  digital  application  and  mobile  technologies  into 
AIT  training:  usability,  utility  or  capability?”  The  final  section  consisted  of  a  wrap-up  question 
to  gather  additional  considerations,  lessons  learned,  or  guidance  that  participants  had  not  yet 
offered. 

Participants 

In  order  to  achieve  a  comprehensive  sample,  participants  with  unique  perspectives  and 
experiences  with  mobile  learning  were  obtained.  Table  1  displays  participants’  positions.  The 
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survey  and  focus  group  questions  were  administered  to  a  total  of  54  participants  from  Fts.  Bliss, 
Gordon,  Leonard  Wood,  Sill,  and  Lee. 


Table  1 

Participant  Position 


Position 

Sample  Size 

Training  Developer 

9 

Student 

14 

Instructor 

8 

Commanders 

2 

Other 

21 

Total 

54 

The  training  developers  were  Anny  civilians  and  one  uniformed  Anny  leader  with 
experience  in  developing  digital  applications  and  interactive  media  instruction  for  handheld 
devices  for  Anny  units.  The  students  and  instructors  had  experience  using  handheld  mobile 
devices  such  as  the  iPhone  and  Samsung  Galaxy  Tablet  during  AIT  or  BOLC.  The  commanders 
had  previous  experience  overseeing  and  implementing  the  integration  of  handheld  mobile 
devices  and  mobile  learning  in  AIT. 

Of  those  participants  that  identified  themselves  as  “Other”,  most  provided  more  specific 
position  titles.  For  example,  rather  than  identifying  themselves  as  a  Training  Developer  they 
said  that  they  were  a  Training  Tech  1702  or  an  Instructional  Systems  Spec  1750.  Others 
provided  their  specific  position  along  with  their  organizational  affiliation,  e.g.,  Chief  of  mobile 
applications  branch,  Lead  Developer  of  mobile  applications  branch,  Deputy  of  technology  for  the 
mission  command  complex. 

Other  factors  that  contributed  to  a  comprehensive  sample  were  the  mix  of  civilian  and 
military  participants,  as  well  as,  participants  rank  or  GS  level.  Table  2  displays  the  breakdown 
of  participants’  military  rank  or  GS  level. 
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Table  2 


Participant  Rank  or  Level 


Civilian 

Enlisted 

Officer 

Rank 

Sample 

Size 

Rank 

Sample 

Size 

Rank 

Sample 

Size 

GS9 

3 

PFC 

3 

2LT 

9 

GS1 1 

7 

PV2 

2 

CPT 

4 

GS12 

3 

SSG 

3 

MAJ 

2 

GS13 

3 

SFC 

4 

LTC 

1 

GSM 

2 

Civ  (unidentified  GS  Level) 

4 

Contractor 

3 

Total  NCO 

12 

Total  Officer 

16 

Total  Civilian 

25 

Total  Military 

28 

Method  of  Analysis 

In  order  to  analyze  this  information,  it  was  essential  to  organize  the  data  according  to  the 
key  considerations  or  factors  that  were  identified  earlier  as  most  important  to  designing  and 
implementing  mobile  learning.  Then,  within  each  factor,  participant  responses,  both  quantitative 
and  qualitative,  could  be  analyzed  to  provide  greater  detail  and  insight  that  would  make  the  final 
list  of  key  considerations  or  factors  comprehensive  and  robust. 

Qualitative  analysis.  The  literature  review  was  used  to  generate  a  list  of  factors  that  are 
integral  to  implementing  training  using  mobile  devices  in  AIT.  There  were  seven  that  were 
initially  identified.  They  are  presented  in  Table  3  with  their  linkage  to  the  original  list  of  factors 
identified  in  the  literature  review. 

Table  3 


Link  Between  Factors  Used  to  Analyze  Data  and  Factors  Identified  in  the  Literature  Review  with 
Descriptions _ 


Factor  from 
Literature  Review 
List 

Factor  Name  in 
Data  Analysis 

Description 

Training  Context 

Training 

Content 

Concerns  the  development  of  applications,  including 
lessons  learned  and  recommendations  on  how  to  develop 
applications  for  handheld  mobile  devices  (e.g.,  expected 
time  investments,  expected  barriers,  application 
development  process  recommendations,  etc.). 

Training  Context 

Training 

Context 

Concerns  the  environment  in  which  the  user  takes  the 
training. 

User 

Students 

Concerns  issues  related  to  students’  use  of  handheld 
mobile  devices  or  digital  applications. 
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Table  3  continued 


User 

Instructors 

Concerns  the  issues  related  to  instructor  attitudes, 
experiences,  preferences,  and  roles  in  handheld  mobile 
device  training. 

Device 

Application 

Concerns  the  application  used  in  the  training,  its  creation, 
its  use,  and  the  potential  value  gained  by  utilizing  the 
application. 

Device 

Device 

Concerns  device-related  issues  to  better  ensure  successful 
integration  of  devices  and  applications  into  AIT. 

Device 

Networking, 
Security,  and 
Implementation 
Issues 

Concerns  networking,  security,  and  implementation  issues 
that  are  important  to  consider  when  devices  will  require 
network  access  and  data  use  and  transmission  are 
involved. 

Creating  a  theme  list  for  analysis.  Subfactors  were  created  for  each  of  the  factors  using 
the  literature  review  and  subject  matter  expertise  gained  from  the  data  collection.  For  example, 
the  “device”  factor  contained  subfactors  such  as,  device  capabilities,  portability,  and  durability. 
These  subfactors  were  created  to  focus  more  closely  on  specific  issues  and  organize  data  within 
each  factor  because  the  factors  were  too  broad,  and  the  issues  within  each  factor  too  complex,  to 
consider  without  further  organization.  The  resulting  list  of  factors  and  subfactors  was  used  as  a 
theme  list  to  examine  transcripts  of  the  interviews,  focus  groups,  and  notes  from  the  data 
collection. 

To  ensure  coding  remained  open  to  new  factors  and  subfactors,  comments  that  did  not  fit 
into  the  factors  or  subfactors  were  coded  as  ‘Other.’  For  example,  an  analysis  of  the  ‘Other’ 
comments  suggested  that  a  new  factor,  labeled  “Institutional  Support”  was  worthy  of  being 
considered  as  a  separate  factor. 

Preparing  for  analysis.  Before  coding  of  the  transcripts  began,  coders  engaged  in  a 
calibration  task.  The  purpose  of  the  calibration  task  was  to  increase  consistency  among  coders  in 
how  they  coded  and  applied  the  factors  and  subfactors  to  comments  from  the  transcripts.  During 
the  calibration  task,  two  coders  coded  the  same  three  transcripts.  Coding  consisted  of  identifying 
blocks  of  content  within  the  transcript,  then  associating  a  specific  factor  and  subfactor  code  with 
that  block  of  content.  Next,  the  coders  compared,  contrasted,  and  reconciled  differences,  as  well 
as  discussed  coding  methods  for  all  three  transcripts.  Following  the  calibration  task,  coders 
proceeded  with  the  coding  of  all  remaining  transcripts. 

Presentation  of  the  results.  23  transcripts  were  coded.  After  coding,  all  of  the  coded 
comments  were  extracted  from  their  transcript  and  organized  by  their  associated  factor. 
Participants’  comments  for  all  of  the  factors  and  their  respective  subfactors  are  reviewed  in  the 
following  section.  Only  the  subfactors  that  contained  enough  participant  comments  for  a 
thorough  understanding  and  sufficient  description  are  summarized  in  the  results.  A  subfactor 
was  considered  to  have  enough  comments  if  there  were  approximately  10  or  more  comments  and 
those  comments  provided  a  clear  understanding  of  the  subfactor.  The  comments  were 
synthesized  into  findings  that  were  used  to  create  a  narrative  for  each  of  the  subfactors.  The 
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narratives  provide  a  summary  of  subfactor  and  include  the  highest  quality  comments  to  display 
similarities  or  agreement  among  participants  and  present  an  accurate  description  for  each 
subfactor. 

During  analysis  it  became  evident  that  two  additional  factors  (i.e.  value  and  cost)  were 
interrelated  to  all  or  most  of  the  factors  and  therefore  should  be  considered  as  overarching  or 
integrated  factors.  These  overarching/integrated  factors  defined  the  objectives  and  constraints 
that  served  as  a  broader  context  for  decisions  in  the  other  factors. 

Quantitative  analysis.  All  of  the  questions  in  the  survey  were  relevant  to  the  factors  that 
were  identified  in  the  literature  review  and  qualitative  analysis  (including  the  purpose  and  cost 
factors).  Table  4  displays  how  many  questions  were  relevant  to  each  of  the  factors. 

Table  4 


Quantitative  Question  Coverage  of  Factors 


Factor 

Number  of 
Questions 

Device  and  Application  Value 

2 

Device  and  Application  Cost 

0 

Application 

2 

Training  Content 

5 

Students 

3 

Device 

4 

Training  Context 

4 

Networking,  Security,  and  Implementation  Issues 

1 

Instructor 

1 

Institutional  Support 

1 

Frequency  distributions  of  participants’  responses  were  examined.  The  frequency 
responses  for  items  are  reported  in  the  results.  The  frequency  percentages  do  not  necessarily  add 
up  to  100%  because  only  the  negative  and  positive  ends  of  the  scale  are  presented.  Midpoint 
frequencies  are  not  presented  because  they  did  not  lead  to  meaningful  findings. 

Results 

The  quantitative  and  qualitative  results  are  reviewed  in  this  section  according  to  their 
associated  factor.  These  findings  were  used  to  refine  the  list  of  factors  and  subfactors  that  were 
used  to  generate  the  decision  guide. 

Mobile/Digital  Application 

This  factor  concerns  the  actual  application  used  in  training,  its  creation,  its  use,  and  the 
potential  value  gained  by  its  employment.  There  were  a  total  of  302  participant  comments 
examined  for  this  factor.  Two  questions  in  the  survey  directly  related  to  the  Application  factor. 
Seven  subfactors  were  identified  that  helped  clarify  the  most  important  considerations  for  this 
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factor.  Table  5  lists  the  sub  factors  and  the  number  of  participant  responses  for  each  sub  factor. 
Each  subfactor  is  discussed  with  supporting  evidence  from  the  survey  and  data  extracted  from 
interviews  and  focus  groups. 

Table  5 


Application  Subfactors  Comments 


Subfactor 

Number  of 
Participant  Comments 

Development 

69 

Implementation 

13 

Information  representation 

14 

User  Interaction 

6 

Usability 

27 

Utility- Value 

25 

Compatibility 

35 

Development.  This  subfactor  concerns  the  development  of  applications,  including 
lessons  learned  and  recommendations  on  how  to  develop  applications  for  handheld  mobile 
devices  (e.g.,  expected  time  investments,  expected  barriers,  application  development  process 
recommendations,  etc.).  There  were  many  comments  related  to  application  development.  The 
comments  regarding  lessons  learned  tended  to  focus  on  very  specific  and  often  technical 
recommendations.  Specific  recommendations  on  how  to  develop  applications  for  handheld 
mobile  devices  focused  on  the  application  development  process,  including  discussions  of  what 
the  process  consists  of  and  recommendations  for  how  development  of  digital  applications  should 
be  conducted. 

Participant  Guidance.  Many  participants  provided  guidance  concerning  the 
development  of  applications,  including  the  following  items. 

•  General  questions  that  developers  need  to  ask. 

o  “Why?”  or  “What  is  the  purpose  of  creating  an  application?” 
o  What  is  the  best  way  to  deliver  content? 
o  What  are  technical  constraints? 

•  Development  Specifications 

o  The  software  that  is  used  to  program  Android  is  8.6  eclipse  for  Manscen  Rapid 
Development  Suite  (MRDS). 

o  Unity  computer  program  programs  for  Apple,  Android  and  RIM  (blackberry), 
o  iOS  is  more  amenable  to  larger  apps  (in  terms  of  storage,  data  size,  etc), 
o  Licensing 

■  The  goal  was  to  get  the  Anny  to  purchase  an  enterprise  and  developer 
license  (free)  and  distribution  license  ($99)  a  year. 

■  For  Google,  you  need  a  license  (one  time  fee  $25). 

■  It  took  14  months  to  get  an  enterprise  license  and  an  additional  three 
months  to  get  the  developer  license. 
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•  Bad  developmental  formats 

o  Applications  that  contain  contractor  proprietary  code, 
o  Applications  without  a  source  file. 

o  Applications  that  require  developers  to  frequently  recreate  code  in  order  to  move 
forward. 

•  User  Interface 

o  Keep  it  simple. 

o  Putting  slides  into  a  phone  isn’t  that  helpful, 
o  Don’t  complicate  the  screens. 

o  Ergonomics:  assume  a  two-hand  grip/hold  or  all  thumb  movement . 
o  Use  pull  down  menus  to  complete  the  forms  with  greater  accuracy;  reduces 
processing  time. 

o  Don’t  have  lots  of  levels  (about  3  tops). 

o  Technical  issue  with  apps:  You  can’t  hover  to  get  more  info  like  with  a  mouse 
o  Take  a  Wikipedia  style  approach  to  TMs,  FMs,  etc  versus  pdf  and  page  turning 
style. 

o  Consider  screen  size  when  developing,  especially  if  requiring  some  interactivity 
between  device  and  user. 

o  Make  graphics  2.5  times  the  maximum  resolution  allowed  so  that  as  device 
display  gets  better,  you  have  built-in  better  graphics  without  rewriting  the  apps. 
o  Use  .pngs  not  .jpegs  for  pictures. 

•  Iconology 

o  Buttons  have  to  reflect  what  it  does, 
o  Make  buttons  easy  and  intuitive  to  understand, 
o  Use  familiar  pictures. 

o  Balance  what  buttons  need  to  be  on  a  screen  at  the  same  time, 
o  Buttons  need  to  be  big  enough  for  user  to  touch, 
o  Use  a  high  quality  graphic  for  buttons. 

•  Developer  training 

o  Many  people  don’t  have  a  programming  background  yet  still  develop 
applications. 

o  There  needs  to  be  a  middle  ground  between  1750  training  developers  and 
programmers. 

o  Developers  are  not  the  only  ones  who  need  training.  Training  Staff  and  cadre  need 
new  skills.  They  need  to  become  familiar  and  comfortable  with  technologies, 
o  The  basic  programmer  can’t  necessarily  do  mobile  apps.  It  takes  more  skills  to 
program  but  has  to  be  combined  with  the  training  objectives, 
o  The  average  training  developer  has  been  in  their  career  for  20  years.  How  do  you 
keep  them  fresh?  You  have  to  take  the  ADDIE  [Analysis,  Design,  Development, 
Implementation,  and  Evaluation]  course.  Professional  development  is  not 
mandated. 
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o  Developers  with  a  computer  science  background  are  needed  and  they  can  teach 
others  how  to  do  it: 

■  Javascript, 

■  HTML  and  Dreamweaver,  and 

■  Visual  Basic  Script 

o  For  Career  field/MOS  53  a  critical  task  is  that  they  must  be  able  to  develop  an 
app. 

o  In  1980s,  used  to  have  an  MOS  for  code  writing.  Currently  there  are  no  MOSs  for 
coding. 

o  MOS  53s  are  as  close  as  they  come. 

Recommendations  on  how  to  develop  applications  for  handheld  mobile  devices. 

Many  participants  provided  examples  of  the  development  process  as  well  as  recommendations 
for  how  the  development  process  should  be  conducted.  Several  participants  described  the  early 
stages  of  the  development  process  focusing  on  the  interaction  that  must  occur  among  various 
stakeholders  in  order  to  produce  the  best  results.  One  participant  stated,  “We  [training 
developers]  need  to  see  the  school  coming  to  us  and  saying  these  are  the  areas  that  we  need  for 
distributive  learning  and  app  development  and  we  help  to  assist  them  to  meet  those  goals.” 
Another  participant  described  a  lack  of  interaction  among  developers  and  subject  matter  experts: 
“School  reps  come  to  developers  and  say,  we  have  great  ideas  for  how  to  use  mobile  apps  in  our 
training,  but  then  hand  over  a  manual  and  ask  for  it  to  be  put  into  an  app.” 

Participants  provided  a  high  level  overview  of  the  development  process  as  provided  below. 

•  Interaction  with  customers. 

o  Levels  of  interaction  varies,  and 
o  Working  on  pipeline  to  deal  with  customers. 

•  Turning  pub  into  ePub:  Discourage  or  redirect  to  other  lower-level  programmers. 

•  Identify  the  wish  list:  Then  prioritize  the  wish  list. 

•  Then  designer  sits  down  and  designs  look. 

o  Design  document  to  see  [storyboard]  how  it  will  look  visually, 
o  Get  story  board  approval,  and 
o  2  weeks  to  3  months  for  this  process. 

•  Time  to  develop  ~  one  month. 

o  Beta  version, 
o  Sponsor  tests,  and 

o  Get  feedback  from  sponsor  [if  there  is  a  sponsor]  before  making  correction 
(suggested  by  a  user  or  by  schoolhouse). 

•  Release  beta  to  market. 

•  Minor  push  updates. 

Early  on,  it  is  important  to  determine  the  level  of  effort  required  and  whether  or  not 
available  government  training  developers  are  capable  of  creating  the  desired  application.  A 
training  developer  thoroughly  distinguished  between  the  types  of  applications  that  are  usually 
within  the  abilities  of  government  training  developers  and  those  applications  that  require 
contractors:  “Skill  sets  for  application  development,  they’re  not  hard  if  you  do  a  couple  of 
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buttons  for  Android.  It  is  not  hard  to  learn.  The  more  complicated  you  get  the  harder  it  gets. 

IMI  level  1  high  school  education.  Level  2  is  push  a  button/High  school.  Level  3  you  need  a  BS 
to  develop  an  app  or  several  years  equivalent  and  programming.  I  wouldn’t  put  someone  in  my 
shoes  off  the  bat.  I  have  10  years  of  practice.  Mine  was  OJT  [on  the  job  training].  If  I  have  that 
with  the  expectation  to  know  coding  in  3  years  great.  You  don’t  have  to  have  degrees.  You 
have  to  have  programming  experience.  Level  4  forget  it.  That  is  scenario  based.  If  you  are 
going  to  create  that  application  you  are  going  to  have  to  know  some  things.  You  are  going  to 
have  the  Instructional  Systems  degree.  It  takes  a  lot  of  talent.” 

Participants  with  application  development  experience  frequently  mentioned  the 
importance  of  developing  templates  for  application  development.  Multiple  training  developers 
requested  or  identified  the  need  for  “standardized  formats/interfaces”  or  “templates.”  One 
training  developer  suggested  the  need  for  “a  GUI  [graphical  user  interface]  for  standard  Anny 
interface,  if  I  have  an  Army  app  I  know  where  it  is.  Big  Army  needs  to  send  that  down  the  pipe. 
As  a  user  I  want  the  things  in  the  same  place.  The  apps  that  I  like,  it’s  because  I  know  the 
buttons  are  in  the  same  place.”  Some  participants  referred  to  templates  created  by  the  Mobile 
Applications  Branch.  Some  training  developers  suggested  that  they  have  moved  beyond  those 
templates:  “We  have  changed  from  the  Mobile  Applications  Branch  template  to  ours.  It  would 
be  good  to  have  a  repository  of  standard  templates.” 

Implementation.  This  subfactor  concerns  the  implementation  of  applications  in  training, 
including  training  on  how  to  use  the  application  and  the  introduction  of  applications  to  students. 
Multiple  comments  suggested  the  importance  of  providing  training  to  “both  instructors  and 
students  on  how  to  use  device.”  Instructors  and  students  need  to  receive  training  on  how  to  use 
devices  and  applications  in  order  to  maximize  the  effectiveness  of  both.  One  training  developer 
described  how  training  can  be  provided  for  instructors  and  to  the  benefit  of  training:  “Frequently 
end  up  creating  a  paper-based  handbook  to  read  for  how  to  use  the  device/app.  Apps  can  teach 
basic  tasks  and  instructors  can  focus  on  more  advanced  things.”  An  EBOLC  student  said, 

“There  was  none  [training].  We  were  issued  the  tablets.  They  didn’t  say  don’t  do  this.  They 
said  play  with  them.  There  was  no  one  who  said  this  is  new  to  you  and  those  people  didn’t  use 
the  tablets.  [It  would  be  beneficial]  if  they  could  have  spent  2  hours  on  familiarization  with  the 
functions  and  apps,  capabilities  and  how  to  download  apps.” 

Students  and  instructors  displayed  negative  attitudes  toward  applications  if  the 
applications  were  not  effectively  demonstrated:  “There  were  some  apps,  but  no  one  went  over 
them.  They  were  just  there  as  junk  apps.  If  we  had  an  explanation  or  they  put  something  useful 
on  it  that  might  help.  The  junk  is  just  more  to  sift  through.” 

Information  presentation:  order  of  presentation  and  media  selection.  This  sub  factor 
concerns  how  infonnation  should  be  presented  in  applications.  Topics  within  this  sub  factor 
include  considerations  for  text  based  vs.  graphic  based  vs.  audio  based  infonnation  presentation, 
the  use  of  scaffolding,  or  linear  vs.  adaptive  user  progression  through  the  application.  Linear 
presentation  consists  of  a  fixed  narrative  or  progression  through  an  application,  whereas  adaptive 
progression  refers  to  applications  where  the  user  experience  is  more  open  or  free  and  progression 
through  the  application  depends  upon  decisions  made  by  the  user.  Participants  frequently 
commented  about  information  acquisition  and  believed  that  different  media  choices  affected  the 
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value  of  the  application.  Participants  tended  to  prefer  rich  media  that  incorporates  video  and/or 
audio  over  applications  that  more  resembled  a  traditional  briefing.  One  training  developer  said 
the  best  fonnats  were  “things  like  video  based  training,  searching  through  forums,  YouTube 
video.”  Another  training  developer  said,  “Process  videos  as  podcasts  and  vodcasts.”  In  terms  of 
text  based  applications  participants  favored  using  “tabs  and  hyperlinks”  to  make  the  application 
more  interactive.  Some  participants  suggested  “incorporating  GPS”;  however  ,  “Apps  that 
require  network  access  are  more  buggy  and  difficult.” 

Some  participants  described  various  types  of  formats,  but  nearly  all  particularly 
mentioned  avoiding  PowerPoint.  One  participant  described  the  desire  to  go  beyond  PowerPoint 
presentations:  “Sustainment  Center  of  Excellence  Mobile  or  (SCoE)  Mobile  is  trying  to 
emphasize  more  than  just  PowerPoint  format.”  SCoE  Mobile  is  the  Army’s  center  of  excellence 
for  mobile  application  creation  and  the  design  of  applications  for  instruction.  Another  training 
developer  discussed  the  failure  of  PowerPoint  to  be  easily  updated  and  therefore  it  becomes 
outdated:  “Delivering  a  PowerPoint  (PPT)  presentation.  Apps  that  allow  you  to  open  a  PPT  are 
not  good  because  you  could  end  up  with  an  outdated  version.  Word  files  are  the  same  way.” 
Another  training  developer  described  a  way  for  linking  applications  to  content  that  would  allow 
for  content  to  be  updated:  “Sponsors  who  want  an  updated  newsletter,  but  don’t  know  how  to  or 
have  an  ability  to  have  a  repository  of  the  content.  Development  is  easiest  when  an  external 
website  or  data  source  is  updated  separately  from  the  app,  but  the  app  just  knows  where  to  go  get 
the  most  recent  data.” 

Participants  suggested  that  interactive  and  entertainment  aspects  of  applications  could  be 
used  to  motivate  learning.  One  participant  said,  “If  it  looks  like  a  game  then  you  will  have  a 
positive  reaction.  For  example,  modeling  a  block  of  instructions  after  a  first  person  shooter 
game.” 


User  interaction.  This  subfactor  concerns  how  users  will  interact  with  applications. 
Considerations  included  the  necessity  of  voice  interaction,  haptic  interaction  via  a  touch  screen, 
text  entry,  visual  interaction  (e.g.,  viewing  training  multimedia),  collaboration  between  students, 
or  network  interaction.  Despite  the  proliferation  of  apps  available,  the  discussion  of  user 
interaction  suggested  that  the  design  of  innovative  user  interaction  involving  mobile  devices, 
particularly  for  Anny  training,  is  still  in  the  early  stages.  One  training  developer  said  that  they 
were  “looking  at  coming  up  with  a  game  that  allows  students  to  play  against  each  other.” 

Several  training  developers  described  the  difficulty  in  producing  more  complicated  technical 
applications.  One  training  developer  said  development  was  a  “two  or  three  year  process, 
multimillion  dollar  process.  By  the  time  the  simulation  is  done  and  completed,  it  is  obsolete.” 
Another  training  developer  described  user  expectations:  “Soldiers  expected  quality  of  content 
presentation  to  be  at  the  level  of  commercial  gaming  or  simulations  -  while  quality  and  fidelity 
are  important,  the  content  must  be  current  and  accurate.”  The  emergence  of  new  technologies 
and  increased  experience  of  Anny  training  developers  will  eventually  lead  to  technically 
complex  applications  that  allow  multiple  users  to  interact.  However,  the  current  capabilities  and 
cost  realities  make  it  difficult  for  these  types  of  applications  to  be  designed  at  present. 

Usability.  This  subfactor  concerns  the  user  friendliness  (an  aspect  of  usability  which  can 
negatively  impact  user  acceptance  of  an  application)  of  applications  and  other  aspects  such  as 
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application  functionality  and  user  interface.  In  the  survey,  participants  were  asked  to  consider 
what  is  important  for  decision  makers  when  determining  how  to  integrate  digital  applications  and 
mobile  technologies  into  AIT.  Specifically,  participants  were  asked  to  identify  the  importance  of 
usability  of  the  device/application  (e.g.,  user  friendliness,  screen  size,  ease  to  program/set-up, 
etc.).  90.2%  of  participants  responded  that  usability  was  either  important  or  very  important. 

Only  4%  of  participants  responded  that  usability  was  slightly  important  or  not  at  all  important. 

User  friendliness.  Whether  the  application  works  well  or  functions  correctly  was 
mentioned  several  times.  One  student  said,  “The  interface,  if  the  software  is  not  user  friendly  for 
those  who  are  messing  around  with  it,  it  discourages  you  from  playing  around  and  figuring  it  out. 
I  was  excited  at  first,  but  then  it  got  frustrating  fast.”  Several  students  described  the  lack  of  user 
friendliness  as  a  key  detenninant  to  whether  or  not  they  use  the  device.  One  student  describing 
an  application  reference  material  said,  “The  pdf  reader,  I  took  mine  to  be  re-encrypted  and  it 
crashed  four  times.”  Another  student  said,  “I  might  spend  4  or  5  min  looking  for  something  and 
never  get  there.  If  I’m  not  going  to  get  anywhere  I  won’t  use  it.”  A  training  developer  said,  “If 
it  is  not  user  friendly  I  am  not  going  to  use  the  device.  As  far  as  an  app,  I  like  to  make  it  look 
pretty,  but  if  I  am  a  user  I  don’t  care  how  it  looks  as  long  as  it  is  quick.  I  don’t  want  it  hid 
somewhere.  I  am  going  to  get  out  and  do  something  else.” 

The  simplicity  of  applications  in  terms  of  navigating  and  the  ease  of  use  were  discussed 
as  important  to  user  friendliness.  Participants  suggested  that  user  friendliness  was  most 
important  or  among  one  of  the  most  important  considerations  concerning  training  using  mobile 
devices.  One  instructor  described  how  applications  that  lack  simplicity,  intuitive  controls,  and 
timely  application  execution  will  lead  to  a  lack  of  use:  “It  has  to  be  simple.  If  it  takes  a  minute 
to  get  into  with  a  bunch  of  things  to  jump  through,  their  attention  span  is  gone.  The  current 
generation  wants  instant  results  and  gratification.” 

User  motivation.  Participants  described  the  need  to  motivate  trainees  to  use 
applications.  One  instructor  said,  “Because  it  is  not  written  into  the  POI,  it  is  limited  to  the 
Soldiers’  motivation.”  Another  participant  said,  “However,  getting  them  motivated  to  do  it  is 
hard.  It’s  too  complex  and  a  lot  of  them  will  just  give  up.  If  you  broke  it  down  it  would  be  more 
effective,  though  it  has  been  effective.”  Describing  the  challenge  of  getting  trainees  to  engage  is 
self-directed  learning,  one  participant  said  that  “to  get  them  to  want  to  leam  on  their  own  time  is 
hard.” 

Many  participants  commented  on  how  if  applications  were  fun  or  entertaining  then  they 
would  increase  trainees’  motivation  to  use  applications  and  leam.  One  training  developer  said, 
“You  can’t  [always]  make  apps  mandatory,  but  if  you  make  it  fun,  they  will  do  it.”  Another 
participant  said,  “It  has  to  be  entertaining.  They  get  excited  at  first,  but  they  get  bored  quickly. 
You  can  put  it  out  to  them  and  say  why  don’t  you  use  the  device.  None  will  do  it  unless  it  is 
more  entertaining.”  Additionally,  instructors  can  use  applications  and  mobile  devices  to  increase 
classroom  motivation.  One  student  said,  “I  don’t  think  a  lot  of  the  instructors  had  an  issue  [with 
devices]  because  it  was  an  opportunity  to  liven  up  the  classes.  We  think  the  classes  are  dull  so 
I’m  sure  they  think  that.” 

Several  participants  suggested  methods  for  increasing  trainees’  motivation  to  use  mobile 
applications.  One  training  developer  said,  “You  have  one  goal  with  little  tiny  goals.  You  need 
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the  little  rewards.  That  is  what  the  little  games  are  used  for,  to  keep  them  going  along.”  Another 
training  developer  similarly  commented,  “In  application,  replicate  achievement/competition 
component  to  motivate  (Scores,  medals,  etc).”  Another  training  developer  described  the  value  of 
demonstrating  the  training  effectiveness  of  applications:  “Show  them  or  demonstrate  a  capability 
that  enhances  their  effectiveness  and  Soldier  perfonnance.  [Some  app  and  device  tests]  have 
shown  great  improvements  in  training  and  task  performance.  Take  advantage  of  parallel  gaming 
presentations  and  graphics  to  increase  Soldiers’  interest  and  motivation  in  the  training.” 

Utility- Value.  This  subfactor  concerns  the  value  associated  with  applications,  such  as 
improved  motivation  of  trainees,  improved  skill  transfer  to  work  tasks,  reduction  in  errors,  etc. 

In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers  when 
detennining  how  to  integrate  digital  applications  and  mobile  technologies  into  AIT.  Specifically, 
participants  were  asked  to  identify  the  importance  of  utility  of  the  device/application  (i.e.  how 
effective  the  device/application  is  for  doing  what  needs  to  be  done).  98%  of  the  participants 
responded  that  utility  was  either  important  or  very  important.  No  participants  responded  that 
utility  was  slightly  important  or  not  at  all  important.  Participants  described  utility  as  one  of  the 
most  important  aspects  concerning  mobile  learning.  The  usefulness  of  applications  and  devices 
were  discussed  in  tenns  of  their  effect  on  training  and  also  their  overall  cost  effectiveness. 

Training  effectiveness.  Many  participants  described  the  need  for  more  information 
concerning  the  effectiveness  of  mobile  applications  and  devices.  One  participant  said,  “The 
instructors  haven’t  been  convinced  that  it  is  worth  using.”  Another  participant  said,  “If  you  have 
relevant  apps  that  make  you  a  better  Soldier  then  yea  what  instructor  doesn’t  want  that.” 
Specifically,  a  participant  referenced  the  need  for  data  which  demonstrates  that  “you  are  going  to 
[be]  better,  based  on  quantifiable  data.”  Participants  suggested  that  it  was  important  to 
demonstrate  the  effectiveness  of  using  the  mobile  devices  and  applications.  In  discussing 
training  effectiveness  one  participant  suggested  that  data  is  already  demonstrating  that  mobile 
devices  are  effective.  For  example,  a  civilian  with  mobile  learning  experience  said  that  a  mobile 
learning  pilot  of  a  Raytheon  devised  application  for  AD  AIT  -  patriot  missile  AIT  at  Ft  Bliss  - 
resulted  in  completing  class  approximately  2  weeks  early  with  14%  increase  in  grade  point 
average.  Another  participant  indicated  there  had  been  a  “50%  increase  across  board  in  academic 
perfonnance.” 

Cost  effectiveness.  Participants  suggested  that  the  cost  effectiveness  of  applications  and 
devices  was  important.  One  participant  said,  “Cost  is  a  big  factor.  Cost-effectiveness  [how 
much  it  helps  compared  to  how  much  it  costs].”  Numerous  participants  identified  benefits  of 
mobile  devices  that  they  believed  contribute  to  the  cost  effectiveness  of  devices.  One  participant 
said,  “Mobile  devices  could  eliminate  the  need  for  paper,  more  instant  compilation  of  textbook.” 
Another  participant  said,  “I  think  it  can  be  effective.  We  have  10  radars  to  work  with.  Other 
groups  only  have  one  radar.  So  equipment  time  is  limited.  Students  are  going  to  be  getting  a  lot 
of  time.  Giving  students  time  digitally  instead  of  on  paper  will  reduce  learning/training  time  and 
reduce  paper  costs.  The  new  generation  will  learn  it  just  a  little  faster,  not  to  mention  saving 
paper  costs.  We  have  to  print  out  TMs,  manuals.  Now  you  are  doing  one  sum  cost  and  update 
the  software  as  changes  come.”  Finally,  one  participant  suggested  that  for  mobile  devices  to  be 
truly  cost  effective  they  would  need  to  be  used  more  in  training:  “I  would  say  there  is  [return  on 
investment]  ROI,  but  I  would  want  it  to  have  more  for  how  much  they  spend  per  unit.  I  would 
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want  it  to  be  more  than  just  march,  order,  and  emplacement  (MOE)  because  that  is  a  small  piece 
of  our  course.  I  think  the  capability  could  be  expanded  to  other  aspects  of  the  course.” 

Compatibility.  This  subfactor  concerns  the  compatibility  of  applications  with  one  or 
multiple  devices  or  operating  systems.  Participants  wanted  to  make  sure  that  application 
compatibility  was  considered  because  users  have  preferences  for  different  devices  which  contain 
different  programing  languages.  One  student  said,  “I  have  an  iPad.  You  go  down  IOS  or 
Android,  most  apps  are  published  on  both.  If  they  could  get  the  developer  license  they  could  do 
both.”  A  developer  said,  “When  we  were  choosing  phones  we  selected  (Android)  phones 
because  we  can  stick  flash  media  in.  Itunes  uses  the  Itunes  fonnat  so  it’s  not  compatible  with  our 
stock  footage.  That  flash  was  not  user  friendly.” 

Training  Context 

This  factor  focuses  on  the  actual  training  environment.  There  were  a  total  of  18 
participant  comments  for  the  Training  Context  factor.  There  were  four  questions  in  the  survey 
related  to  the  Training  Context  factor.  Three  subfactors  were  identified  that  helped  classify  the 
most  important  considerations  for  this  factor.  Table  6  summarizes  the  sub  factors  and  the  number 
of  participant  comments  for  each  subfactor. 

Table  6 


Training  Context  Subfactors  Comments 


Subfactor 

Number  of 
Participant  Comments 

Classroom 

4 

Field/Range 

7 

During  Personal  Time 

7 

Classroom  context.  This  subfactor  concerns  the  use  of  handheld  mobile  devices  and 
training  applications  in  a  classroom  setting.  In  the  survey,  participants  were  asked  to  identify  the 
effectiveness  of  digital  application  and  mobile  technologies  in  various  training  contexts  common 
to  AIT.  Specifically,  participants  were  asked  to  rate  the  effectiveness  of  digital  applications  and 
mobile  technologies  in  classroom  instruction.  61%  of  participants  responded  that  digital 
applications  and  mobile  technologies  were  either  effective  or  very  effective  in  classroom 
instruction  in  AIT.  14.6%  of  participants  responded  that  digital  applications  and  mobile 
technologies  were  either  ineffective  or  very  ineffective  in  classroom  instruction  in  AIT. 

Participants  suggested  that  both  tablets  and  smartphones  were  appropriate  for  classroom 
training.  Tablets  were  most  preferred  by  our  participants  for  training  conducted  in  the 
classroom.  One  training  developer  said,  “some  needed  smartphones,  while  others  needed  tablets, 
especially  in  the  classroom.”  Participants  mentioned  the  value  of  having  tablets  in  the  classroom 
as  a  reference.  One  participant  appreciated  the  ability  these  devices  have  to  manage  greater 
amounts  of  information  and  number  of  terms.  An  instructor  mentioned,  “The  onus  is  on  the 
student  to  bring  it  with  them.  If  they  have  it,  I  let  them  reference  it.”  The  instructor  also  warned 
that  the  reference  material  has  to  be  easy  to  access  as  the  pace  of  the  course  requires  students  to 
search  quickly  through  materials. 
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Field  context.  This  subfactor  concerns  the  use  of  handheld  mobile  devices  and  training 
applications  in  a  field  or  range  context.  In  the  survey,  participants  were  asked  to  identify  the 
effectiveness  of  digital  application  and  mobile  technologies  in  various  training  contexts  common 
to  AIT.  Specifically,  participants  were  asked  to  rate  the  effectiveness  of  digital  applications  and 
mobile  technologies  in  field  instruction.  70%  of  participants  responded  that  digital  applications 
and  mobile  technologies  were  either  effective  or  very  effective  in  field  instruction  in  AIT. 

17.5%  of  participants  responded  that  digital  applications  and  mobile  technologies  were  either 
ineffective  or  very  ineffective  in  field  instruction  in  AIT.  Additionally,  participants  were  asked 
to  rate  the  effectiveness  of  digital  applications  and  mobile  technologies  in  concurrent  training. 
78.6%  of  participants  responded  that  digital  applications  and  mobile  technologies  were  either 
effective  or  very  effective  in  concurrent  training.  1 1 .9%  of  participants  responded  that  digital 
applications  and  mobile  technologies  were  either  ineffective  or  very  ineffective  in  concurrent 
training. 

Participants  suggested  that  both  tablets  and  smartphones  were  appropriate  for  field 
training.  However,  there  were  some  reservations  as  to  what  types  of  field  training  conditions 
were  appropriate.  Device  durability  was  mentioned  as  those  involved  with  physically 
challenging  field  contexts  questioned  the  ability  to  have  devices  in  the  field.  One  instructor  said, 
“I  couldn’t  take  them  [tablets]  on  my  demo  range.  They  are  banned  from  there  because  I  don’t 
want  them  broken.  Our  exercises  are  hands  on.”  For  the  same  reason,  participants  suggested 
that  mobile  devices  might  be  less  appropriate  for  field  training  exercises.  One  student  said,  “We 
are  not  allowed  to  take  these  to  the  field  at  all.  When  we  go  on  FTXs  [field  training  exercises], 
we  can’t  take  them  because  if  they  get  wet  or  dropped  they’re  done.” 

Several  participants  described  the  value  of  having  mobile  devices  in  the  field. 

Participants  mentioned  that  devices  would  be  helpful  in  delivering  infonnation  and  practice  to 
Soldiers  who  are  waiting  to  receive  more  one-on-one  instructor-led  training  or  hands-on  training 
on  a  piece  of  equipment.  One  participant  detailed  how  this  type  of  training  could  be  conducted: 
“We  break  down  our  AIT  into  modules.  Learn  how  to  drive  the  Heinmet.  Then  break  down  the 
launcher,  then  the  MOE  crew  drills.  If  you  broke  the  app  down  like  that  and  they  followed 
along,  how  many  can  an  instructor  do  at  a  time,  4.  If  I  can  only  do  2  at  a  time,  in  the  bleachers 
they  can  be  learning  what  the  gauges  do.  Then  in  module  2,  [while]  I  am  teaching  2  people,  they 
can  be  sitting  there  practicing.” 

During  personal  time.  This  subfactor  concerns  the  use  of  handheld  mobile  devices  and 
training  applications  to  be  used  in  one's  free  time  as  self-development  (e.g.,  the  barracks).  In  the 
survey,  participants  were  asked  to  identify  the  effectiveness  of  digital  application  and  mobile 
technologies  in  various  training  contexts  common  to  AIT.  Specifically,  participants  were  asked 
to  rate  the  effectiveness  of  digital  applications  and  mobile  technologies  in  infonnal  instruction. 
82.9%  of  participants  responded  that  digital  applications  and  mobile  technologies  were  either 
effective  or  very  effective  in  infonnal  instruction.  9.7%  of  participants  responded  that  digital 
applications  and  mobile  technologies  were  either  ineffective  or  very  ineffective  in  informal 
instruction. 

Several  participants  emphasized  the  value  of  having  the  device  for  review  of  training 
materials,  but  that  in  order  for  self-directed  learning  to  work  the  devices  need  to  be  engaging  and 
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the  materials  need  to  be  relevant  to  training.  One  student  said,  “Not  just  a  task,  but  more  of  a 
mode  of  learning.  When  you  take  home  the  device  at  night  you  aren’t  likely  to  break  out  6 
different  FMs  but  if  it  is  more  interactive  they  are  going  to  be  more  likely  to  open  it  up  and  check 
it  out  again,  to  review  or  reinforce  their  learning.  That  would  cut  back  on  training  time  if  people 
were  understanding  things  faster.”  Participants  described  the  value  of  having  the  device  for  use 
in  the  barracks.  One  training  developer  said,  “They  had  reinforcement  training  after  hours.  This 
was  to  refresh.” 

Training  Content 

This  factor  concerns  the  development  of  applications,  including  lessons  learned  and 
recommendations  on  how  to  develop  applications  for  handheld  mobile  devices  (e.g.,  expected 
time  investments,  expected  barriers,  application  development  process  recommendations,  etc.). 
There  were  a  total  of  79  participant  comments  that  focused  on  this  factor.  There  were  five 
questions  in  the  survey  related  to  the  Training  Content  factor.  Seven  subfactors  were  identified 
that  helped  classify  the  most  important  considerations  for  this  factor.  Table  7  reviews  the 
subfactors  and  the  number  of  participant  responses  for  each  subfactor. 

Table  7 


Training  Content  Subfactors  Comments 


Subfactor 

Number  of 

Participant  Comments 

Objectives 

20 

Content  complexity 

13 

Content  constancy/stability 

5 

Individual/Collective  task 

9 

Resource  Requirements 

11 

Objectives.  This  subfactor  concerns  the  objectives  of  training  according  to  the  POI  and 
course  goals.  The  focus  here  is  on  detennining  what  the  training  is  intended  to  accomplish  (i.e., 
successful  trainees  are  those  who  are  able  to. . .?).  In  the  survey,  participants  were  asked  to 
consider  what  is  important  for  decision  makers  when  detennining  how  to  integrate  digital 
applications  and  mobile  technologies  into  AIT.  Specifically,  participants  were  asked  to  identify 
the  importance  of  the  objectives  of  the  training  (e.g.,  training  objectives  versus  learning 
objectives).  85.4%  of  participants  responded  that  the  objectives  of  the  training  were  either 
important  or  very  important.  4.2%  of  participants  responded  that  the  objectives  of  the  training 
were  slightly  important  or  not  at  all  important.  Additionally,  participants  were  asked  to  identify 
the  importance  of  the  nature  of  the  learning  opportunities  and  what  the  focal  intent  of  the  training 
is  on  (e.g.,  knowledge,  skill  acquisition,  practice,  and  feedback).  91.8%  of  participants 
responded  that  the  nature  of  the  learning  opportunities  and  what  the  focal  intent  of  the  training  is 
on  was  either  important  or  very  important.  2%  of  participants  responded  that  the  nature  of  the 
learning  opportunities  and  what  the  focal  intent  of  the  training  is  on  was  slightly  important  or  not 
at  all  important. 
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Training  and  learning  objectives  need  to  be  well  defined  to  make  development  of  mobile 
applications  easier.  One  application  developer  suggested  that  the  program  of  instruction  (POI) 
needs  to  be  established  for  application  developers  to  begin  creating  an  application:  “Analysis  of 
POI  needs  to  be  done  and  rewritten.  They  [(application  developers]  don’t  have  time  to  do  the 
POI.  Training  developers  do  the  POIs.  They  are  trying  to  work  the  pieces  of  getting  resources 
into  redoing  lessons  plans.”  Also,  once  the  application  is  completed  the  content  needs  to  be 
reviewed  and  tested:  “Issues  with  the  content  itself  [some  of  the  steps  were  incorrect  or 
backwards].  I  don’t  know  if  we  had  a  beta  version.  Need  to  review  the  content  for  accuracy.  It 
is  helpful  to  learn  but  it  has  to  be  the  right  thing.”  Additionally,  training  and  learning  objectives 
need  to  be  defined  with  instructors’  direct  involvement  to  ensure  that  what  is  being  developed 
will  be  of  value  to  them:  “For  it  to  be  effective  it  has  to  be  worked  into  the  POIs  with  the 
instructors.  The  programs  have  to  be  simplified  and  appropriate  for  that  POI.” 

In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers 
when  determining  how  to  integrate  digital  applications  and  mobile  technologies  into  AIT. 
Specifically,  participants  were  asked  to  identify  the  importance  of  the  knowledge,  skill,  and 
psychomotor  requirements  of  the  MOS  being  trained.  8 1 .6%  of  participants  responded  that  the 
knowledge,  skill,  and  psychomotor  requirements  of  the  MOS  being  trained  were  either  important 
or  very  important.  10.2%  of  participants  responded  that  the  knowledge,  skill,  and  psychomotor 
requirements  of  the  MOS  being  trained  were  slightly  important  or  not  at  all  important. 
Participants  suggested  that  the  development  of  applications  and  decisions  concerning  what 
course  content  should  be  considered  for  development  is  MOS  specific.  One  participant  said,  “It 
depends  on  what  it  is  for.  Different  MOSs  would  require  different  things.”  Another  participant 
said,  “Consider  the  nature  of  the  skills,  ability  of  students  in  AIT,  the  rough  and  tumble  is  hard  to 
control.  The  group  that  is  willing  to  sit  and  listen.  The  chemical  school  it  is  chemistry,  some 
physical  science.  You  have  to  do  an  analysis  of  a  course.  One  size  does  not  fit  all.” 

Several  participants  suggested  types  of  military  occupational  specialties  (MOSs)  and 
training  objectives  where  mobile  learning  would  not  be  suitable.  Instructors  mentioned  that  they 
wouldn’t  be  useful  in  field  settings  like  a  demo  range.  Another  participant  suggested  that  the 
hands-on  nature  of  training  made  the  use  of  mobile  devices  less  valuable:  “Ours  [training]  is 
repetition.  The  hands-on  training  is  what  is  going  to  make  them  perform  well  on  their  job.  The 
iPhone  isn’t  going  to  get  them  motivated.  When  I  am  telling  a  Soldier  to  MOE  four  times  a  day  I 
need  him  to  be  motivated.  The  hands-on  is  what  gets  them  through  it,  the  battle  buddy, 
camaraderie,  making  it  a  team  effort.”  However,  several  participants  believed  that  mobile 
devices  would  be  helpful  in  certain  types  of  hands-on  situations.  For  example,  one  training 
developer  said,  “Yes,  some  you  need  to  get  your  hands  [involved  with  equipment],  on  that  stuff 
an  application  won’t  help,  but  if  I  have  a  3D  model  of  the  thing  that  I  can  drill  in  and  look  at  the 
whole  things  without  touching  a  brake  system  (i.e.,  of  a  vehicle).  That  would  be  beneficial.  I 
can  drill  down  and  see  the  parts  without  breaking  the  brakes.  I  keep  that  in  a  step  by  step  app 
and  take  it  with  me.  Then  I  have  it  for  later.  I  can  see  its  usefulness.”  One  instructor  suggested  a 
blended  training  context  that  would  make  mobile  devices  improve  upon  the  hands-on  experience: 
“There  is  no  way  to  replace  hands-on  training.  One  area  that  is  enhanced  is  MOE  training. 
Maintenance,  it  would  be  cheaper  on  an  app  than  hands-on  because  maintenance  costs  for  the 
Patriot  are  unreal.  The  MOE  portion  for  that,  it  is  a  great  tool.  It  puts  a  picture  to  the  steps. 
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Prior  to  this,  we  read  through  the  training  circulars  (TCs),  but  didn’t  get  to  do  it  until  we  got  to 
the  later  hands-on  phases.  They  weren’t  getting  hands-on  until  the  walking  phase.  This  almost 
puts  us  at  a  walking  phase  earlier.  Gives  them  better  visualization  of  what  they  have  to  do  and 
when  they  have  to  do  it.  It  cuts  down  time  when  they  actual  do  the  hands  on  training  during  the 
run  phase.” 

Participants  suggested  several  objectives  that  are  suitable  for  training  with  mobile 
devices.  The  most  basic  objectives  were  things  such  as  doctrine  and  course  material  references. 
One  student  said,  “Field  manuals  and  training  manuals  on  the  phone  would  be  great.”  Another 
participant  suggested  that  technical  MOSs  would  benefit  most  from  mobile  learning:  “More 
technical  MOSs  would  be  better  served  using  technology.  But  all  MOSs  are  getting  more 
technical.”  Several  participants  mentioned  specific  MOSs  or  examples  of  training  objectives 
where  the  integration  of  mobile  learning  could  benefit  training  such  as  medical  MOSs,  Infantry 
and  Squad  tactics/movements,  equipment  diagrams  (e.g,  perhaps  through  augmented  reality), 
military  decision  making  process  (MDMP),  Anny  fonn  familiarization,  Physical  Readiness 
Training,  bridge  crossing  calculations,  rifle  marksmanship  visual  training  aids,  supplement  or 
replace  hip  pocket  training  and  aids,  etc. 

Content  complexity.  This  subfactor  concerns  the  complexity/simplicity  of  content  for 
the  specific  task  being  considered  for  the  employment  of  handheld  devices  and  digital 
applications.  The  issues  of  technical  development  difficulty  and  content  complexity  were 
somewhat  interrelated.  First,  participants  made  suggestions  concerning  how  technical 
development  should  be  approached  concerning  difficulty:  “Look  at  in-house  capabilities  -  as 
apps  get  more  complex,  units  might  not  be  capable  of  doing  this  -  have  to  outsource.”  Another 
participant  suggested  that  applications  should  not  become  too  complex  because  they  are  “not 
meant  to  replace  IMI.” 

Participants  suggested  that  the  type  of  content  selected  for  application  development 
should  be  simple.  One  training  developer  said,  “Apps  work  well  for  simple  skills  that  simply 
take  time  to  learn  (versus  elaborate  explanation  or  more  involved  training).”  Another  training 
developer  said  that  applications  work  best  for  training  on  “equipment,  procedure,  protocol.” 
Similarly,  another  training  developer  said,  “Standardized  procedures  [work  well  as  applications] 
versus  ill-defined  problems  with  lots  of  solutions.” 

Content  constancy/stability.  This  subfactor  concerns  the  constancy/stability  associated 
with  the  content  related  to  a  particular  MOS  or  program  of  instruction  (POI).  For  instance, 
training  in  which  the  operational  environment  rapidly  changes  may  require  continuous  training 
content  updates.  Participants  provided  several  comments  that  suggest  that  it  is  important  to 
develop  applications  for  content  that  is  relatively  constant  or  pennanent.  One  participant 
described  the  challenge  of  remaining  current  while  focusing  on  more  established  training 
content:  “The  longevity  of  information  (life  expectancy  of  infonnation).  It’s  easy  to  get  caught 
up  on  new  shiny  object.  A  lot  of  what  we  have  becomes  obsolete,  so  it  takes  longer  to  update 
application  than  [you  have]  to  use  it.  Certain  tactics  will  always  remain  the  same  (M-16 
breakdown  will  always  be  the  same).  Generic  type  of  training  is  more  useful  (IED  may  not  be 
useful  because  it  constantly  changes).  Applications  can  be  used  for  content  that  doesn’t  change 
routinely.  PT  is  another  example  of  something  that  doesn’t  change.  We  can’t  keep  up  with 
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constantly  updating  changes.  If  we  can’t  keep  up  with  rapid  change,  we  can’t  keep  Soldiers  up 
to  date  with  the  most  recent  iteration.”  Another  participant  described  the  difficulty  in  changing 
an  application  to  keep  pace  with  changes  in  training  content,  it  “. .  .depends  on  what  you  are 
doing.  If  we  are  going  to  make  huge  changes  to  application,  it  is  going  to  take  some  time.  We 
are  very  comfortable  for  using  an  application  for  crew  drills.  Communication  equipment 
(something  that  changes  a  lot)  may  not  be  as  cost-effective  or  practical  for  applications.” 

Individual/Collective  task.  This  subfactor  concerns  whether  or  not  the  application  is  for 
individual  or  collective  tasks.  One  of  the  most  frequent  subjects  that  participants  commented  on 
was  the  desire  for  interactive,  collective  learning  using  mobile  applications.  One  training 
developer  said,  “Mobile  devices  can  allow  for  more  realistic  scenarios  in  distributed  teams.” 
Another  participant  said,  “Number  one  thing  that  was  asked  for  was  the  interactive  play.  All  of 
our  equipment  requires  more  than  one  individual  crew  member.  If  we  could  have  one  person  on 
one  phone  driving  and  the  other  person  navigating,  it  would  have  been  more  realistic.  In  the  real 
world  there  is  constant  interaction.  They  are  used  to  interfacing  digitally  and  it  would  make  the 
training  more  realistic.”  Participants  tended  to  value  the  ability  to  collaborate  and  displayed  a 
desire  to  work  in  collaborative  training  environments.  One  student  said,  “Forums  would  be  cool 
where  you  could  post  things  live  and  go  over  things  with  those  in  your  group.  That  would  be 
very  beneficial.” 

Participants  did  suggest  certain  limitations  to  developing  applications  for  collective 
training.  One  instructor  described  the  cost  of  developing  applications  that  are  collaborative  or 
interactive:  “Working  with  others  (i.e.,  crew  drills).  If  you  are  working  with  2  or  3  other  people. 
We  requested  a  multiplayer  app,  but  if  our  budget  limits  us  to  do  single  player,  then  we  can’t  do 
crew  drills,  we  have  to  play  with  a  computer.  Human  interactions  are  more  difficult  to  simulate 
through  an  avatar.  If  I  am  training  with  a  computer  and  then  move  to  a  person  it  will  be 
different.  I  have  to  get  an  experience  of  what  it  is  like  to  work  with  that  person.”  Another 
instructor  described  training  context  where  individuals  have  defined  roles  and  suggested  that  an 
understanding  of  individual  tasks  and  responsibilities  are  required  before  moving  on  to  collective 
learning:  “Single  player  simulations  make  people  learn  their  own  job,  but  if  you  use  multiplayer 
apps,  people  may  not  be  focused  on  learning  their  own  task.” 

Resource  requirements.  This  subfactor  focuses  on  the  resource  requirements  (e.g., 
equipment  or  range)  associated  with  training.  As  an  example,  training  that  necessitates  the  use 
of  special  equipment  would  specify  this  equipment  as  a  resource  requirement  for  effective 
training.  In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision 
makers  when  determining  how  to  integrate  digital  applications  and  mobile  technologies  into 
AIT.  Specifically,  participants  were  asked  to  consider  the  importance  of  the  requirements  of  the 
specific  task  being  trained  (e.g.,  some  tasks  may  require  the  use  of  a  student’s  hands  making  it 
difficult  to  hold  the  device  and  perform  the  task).  88%  of  participants  responded  that  the 
requirements  of  the  specific  task  being  trained  were  either  important  or  very  important.  6%  of 
participants  responded  that  the  requirements  of  the  specific  task  being  trained  were  slightly 
important  or  not  at  all  important.  Additionally,  participants  were  asked  to  identify  the 
importance  of  the  availability  of  training  resources,  especially  operational  equipment,  ranges, 
and  simulators.  87.8%  of  participants  responded  that  the  availability  of  training  resources, 
especially  operational  equipment,  ranges,  and  simulators  was  either  important  or  very  important. 
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2%  of  participants  responded  that  the  availability  of  training  resources,  especially  operational 
equipment,  ranges,  and  simulators  was  slightly  important  or  not  at  all  important. 

Most  of  the  comments  concerning  resource  requirements  described  the  utility  of  mobile 
devices  in  a  blended  learning  environment  to  make  up  for  a  shortfall  in  resources,  such  as 
equipment.  One  participant  said,  “The  crawl  phase.  There  is  not  enough  equipment  for  hands  on 
as  much  as  we  want.  Getting  the  equipment  is  usually  the  biggest  problem.  Mobile  apps  help 
with  that.  It  will  save  us  money  on  maintenance  costs.  If  we  can  avoid  some  of  the  crawl  costs 
it  will  be  a  plus  for  the  Army.”  Lastly,  an  instructor  described  how  mobile  devices  could 
overcome  training  delays:  “One  example  to  add  on  was  that  last  week,  the  class  was  scheduled 
to  do  hands  on  training  for  MOE  but  it  was  inclement  weather,  so  we  had  them  come  in  and  train 
via  the  mobile  apps.  The  next  day  they  were  scheduled  for  the  hands-on  portion  [but  they 
couldn’t  because  of  the  weather]  and  it’s  like  they  were  training  [when  they  used  the 
application].  Normally,  this  would  make  you  behind  schedule  but  they  remained  on  schedule.” 

Students 

This  factor  concerns  issues  related  to  students’  use  of  handheld  mobile  devices  or  digital 
applications.  There  were  a  total  of  46  participant  comments  for  this  factor.  There  were  three 
questions  in  the  survey  directly  related  to  this  factor.  Two  subfactors  were  identified  that  helped 
classify  the  most  important  considerations  for  this  factor.  Table  8  displays  the  subfactors  and  the 
number  of  participant  responses  for  each  subfactor.  Each  subfactor  is  listed  below  with 
supporting  evidence  extracted  from  interview  and  focus  group  data. 

Table  8 


Students  Subfactor  Comments 


Number  of 

Subfactor 

Participant  Comments 

Attitudes/experiences  with  devices/applications 

30 

Readiness  for  self-directed  learning 

10 

Attitudes  and  experiences  with  devices/applications.  This  subfactor  concerns  the 
general  characteristics,  individual  differences,  and  learning  orientation  of  students  in  AIT. 
Examples  of  characteristics,  individual  differences,  and  learning  orientations  include,  but  are  not 
limited  to:  age,  self-efficacy,  and  goal-setting  skills,  respectively.  In  addition,  this  subfactor 
concerns  AIT  students’  attitudes  toward  mobile  devices  (e.g.,  preferences  for  or  against  a 
particular  handheld  mobile  device  or  handheld  mobile  devices  in  general),  previous  experiences 
with  handheld  mobile  devices  (e.g.,  number  of  handheld  mobile  device  experiences  and  valence 
of  those  experiences),  and  readiness  for  self-directed  learning  (e.g.,  are  the  students  motivated  to 
use  devices  and  applications  on  their  own  time  to  enhance  knowledge  and  skills). 

In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers 
to  consider  when  detennining  how  to  integrate  digital  applications  and  mobile  technologies  into 
AIT.  Specifically,  participants  were  asked  to  identify  the  importance  of  the  student’s  level  of 
motivation  and  learning  orientation.  74%  of  participants  responded  that  the  student’s  level  of 
motivation  and  learning  orientation  was  either  important  or  very  important.  10%  of  participants 
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responded  that  the  student’s  level  of  motivation  and  learning  orientation  was  slightly  important 
or  not  at  all  important. 

The  main  characteristic  that  was  discussed  concerned  the  “younger”  generation,  referring 
to  new  and  future  AIT  trainees,  who  are  capable  and  favor  mobile  devices  in  learning  because  of 
their  familiarity  with  mobile  devices.  One  student  said,  “especially  with  this  generation,  [mobile 
learning]  is  very  effective  because  of  the  way  our  generation  has  been  raised  to  operate  is  with 
electronics,  free  time  with  games  or  at  school  with  a  computer.  As  technology  is  developing,  it 
has  great  potential  to  affect  the  quality  of  education  we  receive  at  MOS  in  a  way  we  can  relate  to 
and  therefore  will  remember  better.”  A  commander  contrasted  the  familiarity  of  the  younger 
generation  with  the  difficulty  that  those  who  are  older  might  have  using  mobile  devices:  “The 
group  that  we  are  targeting  is  young  and  are  used  to  the  iPhones  and  smartphones.  For  them,  it  is 
a  good  idea.  However,  we  have  older  folks  who  are  not  used  to  such  devices.  For  them  it  is  not 
the  way  to  go.” 

Comments  suggested  that  mobile  learning  may  reach  out  to  a  segment  of  the  trainee 
population  that  is  more  difficult  to  reach  with  traditional  learning  methods.  One  instructor  said, 
“I  am  not  really  sure  how  to  identify  who  will  be  motivated  to  do  it.  The  ones  that  tended  to  take 
less  notes,  daydream,  and  less  involved  students  used  it  more.  The  ones  that  just  do  enough  to 
pass,  they  were  more  involved  than  others.”  Even  though  students  may  be  more  familiar  with 
the  devices,  the  training  still  needs  to  be  reasonably  easy  to  comprehend.  One  participant 
remarked  that  the  mobile  devices  have  value  “for  those  that  are  videogame  savvy.” 

In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers 
when  determining  how  to  integrate  digital  applications  and  mobile  technologies  into  AIT. 
Specifically,  participants  were  asked  to  identify  the  importance  of  the  student’s  background  and 
experience.  28%  of  participants  responded  that  the  student’s  background  and  experience  was 
either  important  or  very  important.  38%%  of  participants  responded  that  the  student’s 
background  and  experience  was  slightly  important  or  not  at  all  important.  Students  tend  to  have 
familiarity  and  experience  with  mobile  devices,  although  experience  will  vary.  One  student  said, 
“I  didn’t  have  a  computer  until  8th  grade.  These  kids  had  them  in  elementary  school.  99%  have 
a  smartphone.  Their  comfortability  is  higher.”  Age  was  used  as  a  means  for  describing 
differences  in  familiarity  with  devices.  One  student  said,  “The  older  generation  didn’t  have  quite 
the  immersion  in  technology  that  we  did.  There  was  an  older  individual  who  was  a  little  slow  on 
the  uptake,  but  this  individual  eventually  got  it.  Some  people  who  have  already  been  in  the 
military  a  while  may  have  trouble,  but  the  younger  generation  Soldiers  are  likely  to  pick  up  on  it 
faster.”  Familiarity  with  devices  was  also  identified  as  related  to  attitudes  toward  devices.  One 
participant  said,  “In  my  class  I  had  a  wide  spectrum  of  those  who  use  it  a  lot  and  those  who 
haven’t  used  it  at  all.  I  think  familiarity  is  important.  The  people  who  have  not  used  it  are  not 
familiar  with  the  technology  (not  tech  savvy).” 

In  addition  to  experience  with  device,  some  participants  displayed  negative  attitudes 
toward  devices  due  to  the  cost  of  the  device.  Some  participants  discussed  negative  issues 
regarding  responsibility  for  the  device.  One  training  developer  said,  “Some  students  don’t  want 
to  sign  for  the  equipment  and  take  the  risk  of  breaking  or  losing  them.”  Device  cost  and 
responsibility  for  the  device  were  concerns  mentioned  alongside  the  fragility  of  devices.  Despite 
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concerns  over  the  device,  if  precautions  were  taken  to  increase  device  durability  and  Soldiers 
saw  the  value  of  devices,  then  they  were  responsible  for  the  devices.  One  training  developer  said 
there  was  a  “fear  that  devices  were  fragile.  Found  if  Soldiers  see  military  utility  of  the  devices, 
they  will  take  care  of  them  (because  they  value  them).  Added  low-end  covers  and  screen 
protectors;  but  relied  mainly  on  Soldiers’  common  sense.  Out  of  100  devices  used  at  Bliss,  only 
2  destroyed  or  damaged  in  2  yrs  -  devices  were  still  usable  with  cracked  screens.” 

Readiness  for  self-directed  learning.  This  subfactor  concerns  the  amount  of  experience 
AIT  students  have  prior  to  using  a  mobile  learning  device  in  AIT.  In  the  survey,  participants 
were  asked  to  consider  what  is  important  for  decision  makers  when  detennining  how  to  integrate 
digital  applications  and  mobile  technologies  into  AIT.  Specifically,  participants  were  asked  to 
identify  the  importance  of  the  maturity  level  of  students.  60%  of  participants  responded  that  the 
maturity  level  of  students  was  either  important  or  very  important.  14%  of  participants  responded 
that  the  maturity  level  of  students  was  slightly  important  or  not  at  all  important.  Student 
maturity  level  is  related  to  the  level  of  responsibility  and  seriousness  that  are  required  in  order  to 
be  ready  for  self-directed  learning.  The  individual  characteristics  and  attitudes  of  trainees  would 
suggest  that  individuals  are  ready  for  SDL.  Additionally,  a  cultural  shift  to  using  mobile  devices 
would  normalize  the  use  of  devices  in  training.  One  training  developer  said,  “Good  modeling.  If 
more  people  start  to  use  them,  if  influential  people  sell  the  use  of  mobile  devices,  it  will  help  to 
overcome  hesitation.  Seeing  what  devices  can  do,  becoming  a  believer.  Peers  showing/teaching 
people  how  to  use  devices.” 

Participants  recommended  training  to  help  integrate  devices  and  prepare  trainees  for  the 
responsibility  of  SDL.  One  training  developer  said,  “The  solution  -  train  Soldiers  on  how  to  use 
devices  and  when;  as  well  as  when  not  to  use  them  (but  keep  restrictions  reasonable  and 
consistent  -  with  appropriate  follow-through).  Personal  experience  indicated  that  Soldiers  did 
not  abuse  privilege  and  access,  rather  they  were  more  able  to  identify  additional  means  to  exploit 
the  technology  for  mission  and  training.”  Another  commander  described  how  device  training 
can  improve  user  attitudes:  “For  AIT,  we  provided  them  with  a  crash  course  on  how  to  use  the 
phone.  Some  have  caught  interest  in  liking  it.  Someone  younger  who  likes  it  if  you  stick  them 
with  someone  older  then  they  may  like  it.  If  they  don’t  like  it  though,  then  they  won’t  use  it. 
Perhaps  pairing  individuals  who  are  comfortable  with  those  who  are  not  might  help.” 

Instructors 

This  factor  concerns  the  issues  related  to  instructor  attitudes,  experiences,  preferences, 
and  roles  in  using  handheld  mobile  devices  in  training.  There  were  a  total  of  47  participant 
comments  that  focused  on  the  Instructors  factor.  There  were  two  questions  in  the  survey  related 
to  this  factor.  Two  subfactors  were  identified  that  helped  classify  the  most  important 
considerations  for  this  factor.  Table  9  summarizes  the  subfactors  and  the  number  of  participant 
responses  for  each  subfactor.  Each  subfactor  is  discussed  below  with  supporting  evidence 
extracted  from  interview  and  focus  group  data. 
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Table  9 

Instructors  Subfactors  Comments 

Number  of 

Subfactor 

Participant  Comments 

Experience/attitudes  toward  device 

26 

Role  in  Training 

9 

Experience/attitudes  toward  device.  This  subfactor  concerns  the  experiences  and 
attitudes  that  instructors  have  toward  handheld  mobile  technologies  and  their  use  in  AIT.  In  the 
survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers  when 
detennining  how  to  integrate  digital  applications  and  mobile  technologies  into  AIT.  Specifically, 
participants  were  asked  to  identify  the  importance  of  the  background  and  qualifications  of  the 
instructor  conducting  the  training.  72.5%  of  participants  responded  that  the  background  and 
qualifications  of  the  instructor  conducting  the  training  was  either  important  or  very  important. 
13.7%  of  participants  responded  that  the  background  and  qualifications  of  the  instructor 
conducting  the  training  was  slightly  important  or  not  at  all  important. 

Instructor  attitudes  varied  regarding  their  opinion  of  mobile  devices  and  mobile  learning. 
Many  instructors  saw  value  in  mobile  learning.  One  participant  said,  “All  instructors  seem  to 
agree  that  the  app  itself  is  great.”  One  training  developer  identified  common  skepticisms  among 
some  instructors:  “[instructors]  see  mobile  devices  as  temporary  flavor  of  the  month.” 
Participants  pointed  out  the  comfort  that  instructors  had  with  traditional  instruction  and  that 
inexperience  was  a  source  of  skepticism  regarding  the  potential  benefits  of  mobile  learning.  One 
participant  said,  “The  biggest  barrier  is  the  older  generation  wanting  to  do  things  the  way  they 
[the  instructors]  have  always  done  them.  It’s  hard  to  get  instructors  out  of  their  comfort  zone.” 

The  biggest  issue  that  was  raised  was  a  general  resistance  among  instructors  to  change. 
One  commander  said,  “Like  anything  else,  the  initial  change  is  difficult.  A  lot  of  my  instructors 
do  not  like  the  change.  I  think  they  feel  that  they  are  being  replaced  by  a  phone.  Initially  they 
are  pushing  back  and  they  don’t  get  to  see  the  bigger  picture.  It  is  helping  a  lot  of  people.  Yes  it 
does  cost  money  like  anything  else.  But  once  it  is  implemented,  they  will  accept  it  and  move 
forward.”  One  participant  identified  the  source  of  instructor  resistance  to  change  as  “related  to 
level  of  comfort  with  technology.  If  an  instructor  is  set  in  their  ways,  they  might  ask,  ‘Why  deal 
with  it?”’  Some  already  think  they  are  successful  doing  it  how  they’ve  done  it.” 

Several  participants  suggested  fear  of  the  unknown  or  irrational  fear  as  a  barrier  to 
improving  instructor  attitudes  about  mobile  devices  and  learning.  One  participant  identified  a 
“fear  of  computers”  as  a  potential  source  of  concern,  while  another  suggested  that  there  is  “fear 
that  instructors  will  be  lost  because  phones  can  replace  people.”  The  most  frequently  discussed 
fear  was  the  fear  that  instructors  would  lose  control  over  infonnation.  One  training  developer 
described  it  as,  “fear  of  [the]  unknown;  a  cultural  change  especially  among  NCOs,  infonnation 
has  always  been  power  -  smart  phones  gives  everyone  access  to  information,  which  challenges 
the  current  status  quo  of  control.”  Another  training  developer  said,  “Soldiers  set  up  their  own 
communication  groups  to  share  and  exchange  information;  some  NCOs  resisted  or  reacted 
negatively  due  to  a  lack  of  control  of  the  exact  information  being  shared  by  the  Soldiers.” 
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Lastly,  a  student  identified  the  rationale  behind  this  fear,  “The  NCOs’  power  is  by  knowing  more 
than  the  Private.  When  you  give  the  Private  something  where  he  knows  more,  the  NCO  is  not 
going  to  use  it.” 

Perceived  benefits  to  instructor.  Participants  suggested  that  it  was  important  to  get 
buy-in  from  instructors  in  order  to  implement  mobile  learning  effectively.  One  training 
developer  recommended,  “We  need  a  PR  campaign  to  get  buy-in.”  One  participant  identified 
that  the  major  obstacle  is  that  “instructors  need  to  be  motivated  and  capable.”  Several 
participants  made  suggestions  that  may  increase  instructor  motivation  and  capabilities 
concerning  mobile  learning. 

The  chief  method  for  increasing  instructor’s  motivation  to  use  the  device  was  to  clearly 
demonstrate  the  effectiveness  of  mobile  learning.  One  commander  said,  “It  is  mainly  explaining 
to  them  the  benefits  of  the  change.  It  is  also  important  to  let  them  know  that  it  is  not  going  to 
help  everybody,  but  it  will  help  some.”  Several  participants  identified  a  change  in  instructors 
after  they  were  able  to  see  the  capabilities  or  value  of  mobile  devices  in  training.  For  example, 
one  instructor  said,  “I  did  what  I  was  told,  but  I  wasn’t  completely  sold  until  the  one  inclement 
weather  class  where  they  couldn’t  use  the  hands  on  equipment  and  I  saw  that  students  were  able 
to  grasp  concepts.  Then  I  was  completely  sold  and  I  bought  into  it.”  Another  student  participant 
said,  “[instructors]  were  skeptical  in  the  beginning  but  after  watching  us  they  started  to  see  the 
value.  They  saw  us  advancing  further  because  we  were  able  to  get  that  base  knowledge.” 

Lastly,  one  participant  described  how  impressing  the  instructors  with  the  device  and 
application’s  capabilities  could  increase  buy-in:  “[Instructors]  putting  it  on  their  computer  and 
being  able  to  show  a  class  a  simulation  based  video  that  can  be  manipulated  (ex.  Google-Earth 
type  of  environment),  see  different  things  for  maintenance.  It  would  be  useful  and  gain  a  wow 
factor.” 


In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers 
when  detennining  how  to  integrate  digital  applications  and  mobile  technologies  into  AIT. 
Specifically,  participants  were  asked  to  identify  the  importance  of  the  instructor’s  knowledge  of 
mobile  device  capabilities.  82.4%  of  participants  responded  that  the  instructor’s  knowledge  of 
mobile  device  capabilities  was  either  important  or  very  important.  5.9%  of  participants 
responded  that  the  instructor’s  knowledge  of  mobile  device  capabilities  was  slightly  important  or 
not  at  all  important. 

Participants  identified  negative  instructor  attitudes  which  would  suggest  that  it  may  be 
difficult  to  initially  familiarize  instructors  to  devices.  For  example,  one  participant  said, 
“instructors/cadre  are  busy  people”  and  another  participant  said  instructors  “will  see  it  as  extra 
work  to  learn.”  This  was  contrasted  by  another  participant  who  said,  “sometimes  phone  can  help 
because  it  frees  up  instructor  workload.” 

Participants  described  the  importance  of  familiarizing  Instructors  to  devices  and 
suggested  methods  for  doing  so.  One  training  developer  said,  “Give  the  instructor  the  device  and 
let  them  play  with  it.”  Another  training  developer  said,  “Instructors  should  largely  buy  in  on 
their  own  (newer  instructors  are  more  comfortable  with  technology  and  willing  to  use  it).  This 
really  points  to  the  importance  of  their  past  experience,  especially  with  smart  devices.”  Other 
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participants  suggested  that  initial  device  orientation  and  training  should  be  more  involved.  One 
training  developer  said,  “[You]  can’t  just  give  out  smartphones  and  expect  them  to  be  used  or 
embraced.”  An  instructor  said,  “Make  sure  the  instructors  are  competent  on  the  piece  of 
equipment.  Rather  than  a  5  min  spiel  when  they  give  it  to  us  they  need  to  teach  us  the 
capabilities  and  limitations.  That  makes  us  more  likely  to  use  it  and  the  students  to  use  it.  It 
doesn’t  have  to  be  a  course,  just  a  30  min  sit  down,  FAQ  [frequently  asked  questions],  I  am 
decrypting  and  it  won’t  open,  what  do  I  do.  That  way  when  we  are  hit  with  that  we  know  an 
answer.  If  they  know  we  are  confident,  students  can  come  to  us  and  if  we  are  using  it,  they  are 
using  it.  That  would  increase  the  efficacy  of  it.”  Due  to  the  varied  level  of  experience  with 
devices  and  how  busy  instructors  are,  one  participant  suggested  that  instructors  who  need 
training  to  use  mobile  devices  should  be  allowed  to  “follow  his  own  training  calendar  on  it.” 

This  would  allow  them  to  take  training  at  their  own  pace  and  fit  it  into  their  schedule. 

Many  participants  noticed  that  once  instructors  became  familiar  with  the  device  their 
perception  of  the  value  of  the  device  increased.  One  instructor  said,  “After  seeing  the  true  power 
and  the  bigger  picture,  and  why  we  are  trying  to  push  IMI  [interactive  multimedia  instruction] 
they  will  buy  in  to  it.”  A  student  said,  “We  saw  their  opinions  change.  At  first  they  were  just 
compliant  but  then  two  to  three  days  in,  they  started  to  see  the  value.  We  had  a  leg  up  on  others. 
They  told  us  to  keep  running  with  it  because  it  seemed  like  it  was  working.  We  were  able  to  go 
faster  because  we  had  a  base  underneath  us.” 

Participants  also  suggested  that  instructors  would  have  more  buy-in  and  that 
implementation  would  be  improved  if  instructors  were  involved  in  developing  applications.  One 
training  developer  said,  “[There]  has  to  be  a  combination  of  requirement  to  support  it  and  high 
level  support  of  the  instructor’s  ability  to  shape  the  training  around  the  device.” 

Role  in  training.  This  subfactor  concerns  how  instructors  expect  to  use  the  handheld 
mobile  device  in  training.  Examples  of  device  roles  include  supplementing  a  lecture,  as  a 
communication  device  for  instructors  to  contact  students,  or  use  as  a  demonstration  or  practice  of 
necessary  skills  (i.e.,  a  simulation).  In  the  survey,  participants  were  asked  to  consider  what  is 
important  for  decision  makers  when  determining  how  to  integrate  digital  applications  and  mobile 
technologies  into  AIT.  Specifically,  participants  were  asked  to  identify  the  importance  of  the 
instructor’s  ability  to  use  mobile  devices  along  with  instructional  practices.  92%  of  participants 
responded  that  the  instructor’s  ability  to  use  mobile  devices  along  with  instructional  practices 
was  either  important  or  very  important.  2%  of  participants  responded  that  the  instructor’s  ability 
to  use  mobile  devices  along  with  instructional  practice  was  slightly  important  or  not  at  all 
important.  Additionally,  participants  were  asked  to  identify  the  importance  of  the  ability  for 
cadre/instructors  to  control  certain  features  of  the  device.  56.9%  of  participants  responded  that 
the  ability  for  cadre/instructors  to  control  certain  features  of  the  device  was  either  important  or 
very  important.  19.6%  of  participants  responded  that  the  ability  for  cadre/instructors  to  control 
certain  features  of  the  device  was  slightly  important  or  not  at  all  important. 

The  participants  suggested  that  in  order  to  fully  understand  the  role  of  mobile  devices  in 
training,  instructors  need  to  be  trained  on  how  to  integrate  the  devices  into  their  instruction.  One 
instructor  said,  “With  this  application  itself,  we  probably  should  have  had  a  thorough  forum  with 
the  instructors  and  developers  to  make  sure  everyone  is  prepared  on  the  same  page.  Spending 
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time  troubleshooting  and  reacting  is  time  that  could  be  used  elsewhere.  We’ve  had  phones  crash. 
If  we  had  instruction  during  a  lunch  hour  it  would  benefit  instructors.” 

Participants  identified  the  value  of  having  instructors  use  the  mobile  devices  in 
conjunction  with  hands-on  training.  For  example,  one  student  said,  “I  can  see  it  making  the 
teacher’s  job  a  lot  easier.  They  can  tell  us  to  run  through  an  app  on  the  phone  to  practice.” 
Participants  thought  that  mobile  devices  would  be  particularly  effective  in  instances  when  there 
is  a  shortage  of  equipment.  One  student  said,  “Especially  here  where  the  equipment  is  limited 
during  MOE,  you  are  sitting  and  waiting,  you  can  spend  your  time  doing  these  apps  during  down 
time.”  Though  participants  mentioned  the  value  of  mobile  devices  in  preparing  for  hands-on 
training,  participants  also  warned  against  relying  too  heavily  on  the  mobile  devices.  One  student 
said,  “I  have  heard  some  instructors  say  when  everything  falls  apart  there  is  still  a  way  to  do  it. 

If  you  are  so  focused  on  technology,  you  won’t  know  how.  They  will  be  deer  in  headlights. 

They  don’t  want  to  do  the  harder  way,  but  you  have  to  because  everything  is  shut  down.”  This 
comment  describes  a  fear  that  Soldiers  would  become  overly  reliant  on  technology  and  that 
without  the  technology,  they  would  not  be  able  to  function.  These  comments  suggest  that  mobile 
devices  should  not  replace  hands-on  training;  rather,  mobile  applications  could  help  augment 
hands-on  training. 

Participants  suggested  that  mobile  devices  would  be  valuable  to  instructors  who  were 
able  to  take  advantage  of  device  networking  and  trainee  collaboration.  One  student  suggested, 
“If  they  [instructors]  could  use  Wi-Fi  or  even  connect  us  through  blackboard.  Things  like  that 
can  provide  real  feedback  to  the  instructor.”  Another  student  said,  “Setting  up  a  network  so  you 
[other  instructors]  could  monitor  what  is  going  on,  on  the  tablets.  He  [an  instructor]  could  do  a 
check  on  learning,  having  people  buzz  in  on  an  answer.  We  had  systems  like  that  in  college.” 
Lastly,  a  training  developer  requested  other  measures  that  instructors  could  use  to  monitor  their 
students,  “Still  need  measures  and  processes  in  place  that  provide  feedback  to  the  instructor 
(frequency  of  use,  duration  of  self-study,  areas  of  difficulty,  etc.). 

Institutional  Support 

This  factor  concerns  institutional  support  in  utilizing  handheld  mobile  technologies  for 
training.  There  were  a  total  of  24  participant  comments  regarding  this  factor.  There  was  one 
question  in  the  survey  related  to  the  Institutional  Support  factor.  Two  subfactors  were  identified 
that  helped  classify  the  most  important  considerations  for  this  factor.  Table  10  displays  the 
subfactors  and  the  number  of  participant  responses  for  each  subfactor.  Each  subfactor  is 
discussed  below  with  supporting  evidence  extracted  from  interview  and  focus  group  data. 

Table  10 


Institutional  Support  Subfactors  Comments 


Number  of 

Subfactor 

Participant  Comments 

Broad-based  support  in  Army  culture 

18 

Support  from  Chain  of  Command 

6 
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Broad-based  support  in  the  Army  culture.  This  subfactor  concerns  the  broad-based 
support  in  the  Army  culture  for  embracing  and  using  handheld  mobile  devices  in  AIT  or  training 
in  general.  Many  participants  identified  issues  with  regards  to  broad-based  support  in  the  Anny 
culture.  One  participant  identified  a  way  of  overcoming  challenges  or  barriers  to  implementing 
mobile  learning:  “It  has  to  be  top  down.  The  top  has  to  push  down  that  intent  and  make  it 
happen.  Saying  ‘it’s  great,  make  it  happen’  isn’t  helpful.”  Participants  identified  many 
problems  arising  for  not  having  a  top  down  approach  to  implementing  mobile  learning.  One 
participant,  referring  to  a  lack  of  technical  standards  said,  “We  have  received  pushbacks  saying 
we  can’t  buy  iPhones  because  the  Army  does  not  approve  them.  So  whose  approval  do  you 
need?  The  top  officials  either  need  to  make  the  decision  or  to  give  discretion  to  make  those 
decisions.”  One  instructor  noted  the  difficulty  in  getting  approval  to  change  the  POI  to  include 
the  use  of  mobile  devices:  “Any  POI  change  has  to  be  approved  at  multiple  levels  and  takes  a 
while  for  a  revision  on  POI.” 

Participants  identified  cultural  issues  in  understanding  Anny  goals  concerning  mobile 
learning.  One  training  developer  identified  barriers  to  reaching  the  Anny’s  mobile  learning 
goals:  “The  Network  Enterprise  Center  (NEC)  is  the  last  say  on  this.  They  don’t  understand 
[ALM].  FY14  is  the  pilot  for  testing  [ALM].  The  same  thing  as  the  CSDA  we  are  waiting  on 
rules  to  change  to  stand  up  [ALM]  when  TRADOC  is  the  only  one  interested.  Support  from 
outside  is  not  there.”  Another  training  developer  described  what  the  focus  should  be  moving 
forward:  “What  do  they  want  to  understand  -  main  thing  curiosity,  and  innovation  and  what  your 
limits  are  -  Army  learning  model  2015  is  our  guidepost.  Things  need  to  be  done  in  2012  to 
reach  this  goal  in  2015.  Set  aside  money  and  resources  for  developing  toward  that.” 

Participants  suggested  that  cost  and  time  requirements  for  implementation  were  issues. 
One  participant  described  the  issue  of  time:  “The  biggest  problem  is  time.  We  are  constantly 
moving  and  it  is  hard  to  sit  back  and  focus  solely  on  embracing  the  technology.  I  don’t  think 
there  is  a  problem  with  the  concept  at  all.  Finding  time  to  discuss/understand  mobile  app  use  and 
implementation  is  the  hardest  part.”  Additionally,  participants  mentioned  cost  as  an  issue  given 
the  current  environmental  context.  One  instructor  said,  “Money  issues  are  present.  We  are 
going  to  have  to  tweak  the  programs  to  fit  the  budget.  It’s  all  about  the  money.”  A  training 
developer  said,  “[some  have]  concern  that  apps  will  cut  the  budget,  [that]  will  cause  resistance 
(less  need  for  funding  to  do  same  big  course  if  apps  can  cut  personnel  or  training  time).” 

Support  from  chain  of  command.  This  subfactor  concerns  support  from  the  Chain  of 
Command  for  using  handheld  mobile  devices  in  AIT.  In  the  survey,  participants  were  asked  to 
consider  what  is  important  for  decision  makers  when  detennining  how  to  integrate  digital 
applications  and  mobile  technologies  into  AIT.  Specifically,  participants  were  asked  to  identify 
the  importance  of  leadership  support  for  implementing  mobile  devices  for  training.  90%  of 
participants  responded  that  leadership  support  for  implementing  mobile  devices  for  training  was 
either  important  or  very  important.  2%  of  participants  responded  that  Leadership  support  for 
implementing  mobile  devices  for  training  was  slightly  important  or  not  at  all  important. 

Most  participants  identified  that  commanders  are  buying  in  to  the  idea  that  mobile 
devices  and  their  corresponding  digital  applications  can  be  beneficial  for  training.  One 
participant  described  the  buy-in:  “Our  battalion  commanders  now  really  pushed  and  embraced  it. 
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They  are  the  ones  that  really  embrace  it.  Anything  that  can  help  us  put  a  better  product  out,  they 
are  happy  to  help  with.”  Another  participant  said,  “Higher  chains  of  command  are  happy  to  try 
something  new  that  is  supposed  to  be  beneficial  or  if  it  enhances  training.”  These  comments 
reflect  the  importance  of  demonstrating  that  the  devices  are  effective  at  enhancing  training. 

Participants  also  suggested  the  importance  of  getting  everyone  on  board.  One  participant 
said,  “A  lot  of  support  and  drum-banging  is  needed.  I  am  not  sure  how  long  that  steam  is  going 
to  maintain.  The  chains  of  command  are  currently  diligent,  but  if  you  get  one  person  who  is 
afraid  of  technology  then  you  are  in  trouble.”  A  training  developer  said,  “To  get  buy-in  for  using 
mobile  technologies  you  need  to  have  the  entire  command  saying  you  will  do  this.”  The 
comments  suggest  that  unanimous  support  from  the  chain  of  command  will  help  in  the 
implementation  of  mobile  devices  into  training. 

Device 


The  Device  factor  concerns  device-related  issues  to  better  ensure  successful  integration  of 
devices  and  applications  into  AIT.  There  were  a  total  of  154  participant  comments  for  the 
Device  factor.  There  were  four  questions  in  the  survey  related  to  the  device  factor.  Five 
subfactors  were  identified  that  helped  classify  the  most  important  considerations  for  this  factor. 
Table  1 1  displays  the  subfactors  and  the  number  of  participant  responses  for  each  subfactor. 

Each  subfactor  is  discussed  below  with  supporting  evidence  extracted  from  interview  and  focus 
group  data. 


Table  1 1 

Device  Subfactors  Comments 


Subfactor 

Number  of 
Participant  Comments 

Device  capabilities 

42 

Usability 

12 

Durability 

10 

Portability 

10 

Supportability 

4 

Device  capabilities.  This  subfactor  concerns  the  capabilities  associated  with  various 
handheld  mobile  devices  and  the  value  of  capabilities  for  mobile  learning  in  AIT.  In  the  survey, 
participants  were  asked  to  consider  what  is  important  for  decision  makers  when  determining  how 
to  integrate  digital  applications  and  mobile  technologies  into  AIT.  Specifically,  participants 
were  asked  to  identify  the  importance  of  the  type  of  device  (e.g.,  tablet  versus  smart  phone). 

66%  of  participants  responded  that  the  type  of  device  was  either  important  or  very  important. 

14%  of  participants  responded  that  the  type  of  device  was  slightly  important  or  not  at  all 
important. 

Although  data  from  SME  [subject  matter  experts]  interviews  and  focus  groups  suggested 
how  all  devices  may  have  the  potential  to  support  mobile  learning,  an  analysis  of  the  data  did  not 
yield  a  clearly  preferred  mobile  device.  The  data  indicated  that  some  individuals  clearly 
preferred  mobile  phones:  “I  would  prefer  the  phone  because  of  its  portability.  I  can  take  it 
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anywhere.”  While  others  clearly  preferred  other  mobile  devices  such  as  tablets:  “To  me,  using 
the  phone  seems  kind  of  silly  as  opposed  to  the  tablet,”  or  netbooks,  “We  are  looking  at 
netbooks.  The  advantage  is  they  are  CAC  [common  access  card]  card  ready. .  .all  of  the 
problems  of  encryption  have  gone  away.” 

In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers 
to  consider  when  detennining  how  to  integrate  digital  applications  and  mobile  technologies  into 
AIT.  Specifically,  participants  were  asked  to  identify  the  importance  of  device  capabilities  such 
as  memory,  input/output,  capabilities,  connectivity.  80.4%  of  participants  responded  that  device 
capabilities  such  as  memory,  input/output,  capabilities,  and  connectivity  were  either  important  or 
very  important.  3.9%  of  participants  responded  that  device  capabilities  such  as  memory, 
input/output,  capabilities,  and  connectivity  were  slightly  important  or  not  at  all  important. 

Device  preferences  were  largely  grounded  by  the  interaction  between  device 
characteristics  and  the  training  objectives  and  content.  Specifically,  the  preferred  device 
characteristics  are  detennined  by  the  training  content  and  objectives,  wherein  the  preferred 
device  for  one  situation  may  not  be  the  preferred  device  for  another.  As  an  example,  one 
training  developer  stated,  “Tablets  worked  best  for  leaders  in  field  and  generally  in  the  classroom 
due  to  having  to  manage  greater  amounts  of  information  and  number  of  terms.”  In  contrast,  a 
student  stated,  “[The]  computer  is  better  for  us.  With  apps  and  tabs  [tablets]  it  is  a  singular  path, 
with  a  computer  you  can  do  a  lot  more  with  that  operating  system.  The  tab  [tablet]  is  useful 
when  the  task  is  simple.”  Major  points  of  consideration  related  to  mobile  device  preference  (i.e., 
laptop,  netbook,  mobile  phone,  and  tablet)  are  the  previously  mentioned  device  themes  (i.e., 
device  capabilities  and  usability,  network  connectivity,  device  display  characteristics  and 
features,  and  device  portability).  These  themes  are  discussed  below. 

Usability.  This  subfactor  concerns  the  usability  of  handheld  mobile  devices  and  the 
associated  training  required  for  effective  use  of  devices.  In  the  survey,  participants  were  asked 
to  consider  what  is  important  for  decision  makers  when  detennining  how  to  integrate  digital 
applications  and  mobile  technologies  into  AIT.  Specifically,  participants  were  asked  to  identify 
the  importance  of  the  usability  of  the  device/application  (e.g.,  user  friendliness,  screen  size,  ease 
to  program/set-up,  etc.).  90.2%  of  participants  responded  that  the  usability  of  the 
device/application  was  either  important  or  very  important.  4%  of  participants  responded  that  the 
usability  of  the  device/application  was  slightly  important  or  not  at  all  important. 

In  line  with  the  discussion  regarding  the  relationship  between  device  preference  and 
device  appropriateness  for  training,  the  capabilities  of  a  device  largely  determine  decisions 
related  to  specific  mobile  device  preference  and  usage,  as  well  as  the  types  of  tasks  amenable  to 
mobile  training.  In  support  of  this,  one  student  specified  that  detennining  the  most  appropriate 
tasks  for  mobile  training  depended  on  the  technological  capabilities  of  the  hand  held 
device.When  prompted  with  an  inquiry  related  to  the  importance  of  capability  relative  to 
usability  and  utility,  one  student  stated,  “Capability  [is  the  most  important].  If  it  isn’t  capable  of 
doing  what  we  want,  there  isn’t  a  point  to  it.”  In  contrast,  it  should  also  be  noted  that  the 
usability  of  the  device  was  also  referenced  by  numerous  Soldiers  as  the  most  important  device 
characteristic.  Multiple  Soldiers  expressed  the  sentiment  that  if  a  device  was  not  easy  to  use, 
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instructors  would  not  buy-in  to  instructional  use  of  the  device  and  students  would  fail  to  see  the 
benefit  of  the  device:  “if  they  don’t  see  it  as  usable,  the  rest  won’t  matter.” 

The  remaining  themes  within  the  Device  factor  highlight  specific  device  capabilities  that 
these  Soldiers  were  discussing.  These  themes  relate  to  usability  as  well,  for  if  the  desired 
capabilities  are  not  present,  the  device  is  not  seen  as  useful.  Specifically,  the  network 
connectivity  capabilities,  display  capabilities,  and  device  portability  and  durability  determine 
whether  or  not  a  specific  device  is  useful  for  training,  as  supported  by  Soldier  comments. 

Display.  Numerous  comments  revolved  around  the  issue  of  device  display.  Many 
Soldiers  expressed  concern  over  the  small  size  of  most  mobile  phones:  “As  far  as  the  phone,  the 
screen  never  seemed  big  enough.  I  would  say  a  7-10  inch  screen  on  a  tablet  [is  better].”  A 
training  developer  supported  this  notion  by  saying,  “Screen  size  is  a  big  thing.  In  AIT,  screen 
size,  forget  the  phone.  I  have  thrown  that  idea  out  the  door.  The  tablet’s  screen  size  is 
wonderful.”  However,  one  should  consider  the  relation  of  the  desired  task  to  the  size  of  the 
screen.  Specifically,  for  training  tasks  that  rely  on  visual  displays,  mobile  phones  may  not  be 
optimal.  When  asked  about  barriers  for  implementation  of  mobile  devices  in  AIT,  an  instructor 
responded  saying,  “The  size  of  the  iPhone  was  small.  It  is  hard  to  see  a  real  descriptive  view  of 
the  equipment.  If  we  move  towards  a  table,  it  may  be  more  beneficial.”  Furthermore,  one 
Soldier  stated  that  students  loved  the  tablet  because  of  the  “backlit  screen,”  a  feature  that  is 
important  if  mobile  device  usage  involves  reading  materials  outside. 

In  addition  to  the  size  of  the  display  for  visualization  purposes,  training  tasks  that 
leverage  the  interactivity  of  touch  screens  must  also  consider  the  size  of  the  mobile  display.  As 
one  instructor  stated,  “If  someone  has  big  fingers,  the  phone  won’t  do  what  you  want  it  do  to.” 

A  student  provided  a  similar  statement  saying,  “Having  to  tap  the  exact  spot  was  difficult.  A 
little  more  tolerance  may  reduce  the  need  for  a  large  screen.  We  knew  where  to  hit,  we  just  had 
to  find  that  pinpoint.”  Further,  multiple  Soldiers  stated  that  the  size  of  the  screen  would 
influence  the  success  of  integrating  mobile  devices  into  training,  stating,  “It  would  be  better  to 
increase  the  size  of  the  device  like  a  tablet.” 

Network  connectivity.  Network  connectivity  was  also  a  main  focal  point  of  many 
statements  regarding  the  use  of  mobile  devices.  Numerous  training  programs  have  the  potential 
to  utilize  networking  capabilities,  or  in  some  cases  necessitate  networking  capabilities  for 
collaborative  training  tasks  or  sending  and  receiving  infonnation,  such  as  field  manuals.  One 
instructor  offered  an  example  during  the  interview  process,  stating,  “If  we  could  download 
something  like  blue  force  tracker  where  you  could  get  graphics  maps,  pinpoint  your  location,  use 
it  in  orders  and  blast  that  out  to  other  readers  on  a  blue  tooth  that  would  be  a  good  application  in 
a  FTX.”  It  is  clear  that  understanding  the  network  connectivity  capabilities  of  a  mobile  device  in 
relation  to  intended  use  within  a  training  context  is  essential.  This  notion  was  captured  by  a 
training  developer  who  stated  that  “understand  [ing]  network  requirements  for  full  device 
capability  and  functionality”  were  extremely  important. 

During  our  interviews,  it  became  evident  that  although  numerous  devices  have 
networking  potential,  drawbacks  exist  with  using  certain  mobile  devices  on  the  Army  network. 
Multiple  comments  touted  the  networking  capabilities  of  netbooks  in  relation  to  network 
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connectivity  authority.  Comments  from  multiple  participants  demonstrated  that  the  current 
Anny  networking  infrastructure  supports  the  use  of  netbooks:  “The  advantage  [of  netbooks]  is 
they  are  CAC  card  [capable],”  are  “approved  to  touch  the  .mil  [network],”  “can  be  used  in/out  of 
the  classroom,”  and  eliminate  the  problems  of  encrypting  data;  “all  of  the  problems  of  encryption 
have  gone  away.”  Alternatively,  other  mobile  devices,  such  as  Apple’s  tablet,  the  iPad, 
demonstrate  obvious  network  connectivity  concerns.  Although  the  iPad  has  the  ability  to  access 
networks  in  general,  as  well  as  send  and  receive  infonnation,  one  Soldier  stated  that  “anything 
that  would  go  down  the  iPad  route  is  a  pipe  dream  because  of  the  make  of  the  iPad  vs.  Army 
security.”  This  was  also  true  for  mobile  phone  as  well:  “iPhones  on  the  network  were  not  a 
positive.”  Thus,  networking  connectivity  considerations  are  two-fold.  One,  if  the  training 
content  necessitates  collaboration  or  networking,  it  is  imperative  to  ensure  that  the  device  has 
networking  capabilities.  Two,  if  the  (NEC)  does  not  support  the  networking  use  of  a  certain 
device  (e.g.  iPad),  the  networking  connectivity  capability  of  the  device  itself  is  a  moot  point. 

Durability.  This  subfactor  concerns  the  toughness  and  ability  to  withstand  various 
physical  conditions  of  various  handheld  mobile  devices  in  different  AIT  training  contexts.  In  the 
survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers  when 
detennining  how  to  integrate  digital  applications  and  mobile  technologies  into  AIT.  Specifically, 
participants  were  asked  to  identify  the  importance  of  the  durability  of  the  device/platfonn  being 
used  for  the  application.  84%  of  participants  responded  that  the  durability  of  the  device/platform 
being  used  for  the  application  was  either  important  or  very  important.  2%  of  participants 
responded  that  the  durability  of  the  device/platform  being  used  for  the  application  was  slightly 
important  or  not  at  all  important. 

One  instructor  stated  that  mobile  devices  used  in  field  settings  “would  need  to  be 
durable,”  as  well  as  “waterproof.”  Other  Soldiers  expressed  concerns  over  device  durability 
stating,  “Another  issue  is  if  we  moved  to  tablets,  in  the  field  they  may  not  be  durable  enough. 
You’d  be  dealing  with  fragile  equipment.  You’d  be  facing  things  being  broken.  Sticking  to  a 
smaller,  durable  device  i.e.,  an  iPhone  with  a  case  on  it)  is  probably  better.  I  drop  my  personal 
phone  and  it  is  usually  fine.”  Thus,  it  is  clear  that  device  durability  is  an  important 
consideration,  especially  if  the  device  is  intended  to  be  taken  into  field  environments.  One 
Soldier  stated  that  “ruggedness”  is  an  important  device  consideration  because  “they  [Soldiers] 
are  going  to  be  climbing  on  stuff,  falling  off  stuff,  in  the  dirt,  and  in  the  rain.”  Also,  it  seems  as 
if  tablets  are  viewed  as  less  durable  than  mobile  phones;  “tablets  were  more  fragile  than  phones.” 

Portability.  This  subfactor  concerns  the  ease  of  transport  of  the  device.  The  portability 
of  mobile  devices  is  a  characteristic  that  makes  them  attractive  for  training  use.  When  asked 
what  made  the  mobile  devices  effective,  one  training  developer  responded,  “A  part  was  the  new 
wow  factor.  Students  went  from  carrying  books  around  to  carrying  this  information  all  in  one.” 
Increased  portability  also  means  increased  opportunity  to  engage  in  training  and  development 
activities.  One  indicated  that  the  portability  of  a  mobile  device  over  a  laptop  was  a  clear 
advantage  stating,  “If  a  student  is  motivated,  they  can  use  the  app  during  chow  time.  It  allows 
them  more  training  time.”  Although  some  Soldiers  stated  they  would  prefer  a  phone  over  a 
tablet  when  considering  portability,  other  Soldiers  disagreed  stating  that  although  “tablets  [are]  a 
bit  too  large  and  cumbersome,  I  found  seven  inch  tablets  worked  well  in  the  field,  as  they  fit  into 
cargo  pockets.”  Thus,  smaller  tablets  may  be  seen  as  portable  as  mobile  phones.  However,  it 


56 


seems  clear  that  if  portability  is  desired,  larger  tablets  are  not  afforded  the  portability  of  the 
smaller  size  mobile  phones. 

Supportability.  This  subfactor  concerns  the  ability  and  resources  required  to  support 
various  handheld  mobile  devices  over  time,  including  the  ability  to  maintain  and  upgrade 
devices.  Forethought  and  added  consideration  should  be  given  to  the  development  of 
applications  and  training  as  the  development  of  technology  will  continue  to  outpace  the 
development  of  applications.  When  asked  how  they  account  for  devices  becoming  outdated,  one 
participant  said,  “For  the  basic  [development  that]  we  do,  we  don’t  have  to  keep  pace  with  it  [the 
pace  of  technological  advancement].”  This  comment  suggested  that  less  advanced  applications, 
though  they  will  not  keep  pace  with  technological  capabilities,  will  still  be  supported  by  future 
devices.  A  training  developer  clearly  stated  that  the  “goal  is  to  have  them  consume  our  products 
regardless  of  the  device.” 

Networking,  Security,  and  Implementation  Issues 

This  factor  concerns  the  application  used  in  training,  its  creation,  its  use,  and  its  potential 
value.  There  were  a  total  of  302  participant  comments  that  focused  on  this  factor.  There  was 
one  question  in  the  survey  related  to  the  application  factor.  Seven  subfactors  were  identified  that 
helped  classify  the  most  important  considerations  for  this  factor.  Table  12  lists  the  subfactors 
and  the  number  of  participant  responses  for  each  subfactor.  Each  subfactor  is  discussed  below 
with  supporting  evidence  extracted  from  interview  and  focus  group  data. 

Table  12 


Networking  and  Security  Subfactors  Comments 


Subfactor 

Number  of 

Participant  Comments 

Networking  capabilities,  restrictions,  and  policy  requirements 

36 

Device,  application,  and  data  security 

11 

Student  restrictions 

22 

Device  management 

17 

Networking  capabilities,  restrictions,  and  policy  requirements.  This  subfactor 
concerns  the  networking  capabilities,  restrictions,  and  policy  requirements  associated  with 
connecting  and  using  handheld  mobile  devices  on  a  network.  In  the  survey,  participants  were 
asked  to  consider  what  is  important  for  decision  makers  when  detennining  how  to  integrate 
digital  applications  and  mobile  technologies  into  AIT.  Specifically,  participants  were  asked  to 
identify  the  importance  of  network  availability.  66%  of  participants  responded  that  network 
availability  was  either  important  or  very  important.  10%  of  participants  responded  that  network 
availability  was  slightly  important  or  not  at  all  important. 

The  networking  capabilities  of  the  mobile  devices  have  previously  been  discussed  and, 
thus,  will  not  be  revisited  here.  The  focus  of  this  section  in  the  decision  making  process  is  the 
networking  capabilities  of  the  location  (i.e.,  Army  base),  where  training  is  to  occur.  Networking 
capabilities  are  important  only  to  the  extent  that  the  training  content  in  question  necessitates  the 
ability  to  connect  and  interact  with  a  network  to  be  effective.  As  stated  by  one  training 


57 


developer,  when  devices  cannot  touch  the  .mil  server  and  are  without  a  service  plan/wireless 
network,  “the  apps  for  the  devices  are  great  but  it  is  limited  in  applicability. .  .when  you  give 
them  [students]  no  connectivity,  it  is  a  joke  and  they  [students]  don’t  use  it  [the  mobile  device].” 
There  were  issues  with  NEC  and  Army  networking  rules  and  governance,  with  one  Soldier 
stating  that  NEC  “was  our  biggest  roadblock”  for  implementing  the  use  of  mobile  devices. 
Specifically,  this  Soldier  stated  that  approval  for  the  iPhone  “took  about  6  months.”  Another 
Soldier  added  that  “getting  around  the  NEC  and  their  rules”  was  the  biggest  barrier  to 
implementing  devices  in  AIT. 

Alternatively,  in  some  locations,  Soldiers  and  support  staff  were  permitted  to  build  their 
own  network  “to  get  around  security  issues”  or  use  internal  networks  already  in  place.  This 
ability  seems  to  be  of  paramount  of  importance,  with  one  training  developer  going  so  far  as  to 
say  “until  WIFI  and  3G  happens,  it  is  inconvenient  to  do  mobile  learning.”  For  instance,  when 
attempting  to  train  Soldiers  on  a  task  that  typically  requires  collaboration,  if  mobile  devices  are 
not  capable  of  connecting  to  a  network  to  allow  interaction  between  Soldiers,  the  training  may 
not  be  as  realistic,  and,  thus,  effective.  One  Soldier  stated  that  having  interactive  ability  would 
have  made  the  training,  “more  realistic.”  Thus,  it  seems  important  to  consider  the  network 
capabilities  of  the  installation  in  question,  with  a  specific  focus  on  network  restrictions  and 
policy  requirements.  Although  some  areas  seem  to  have  solutions  to  NEC  rules  and  governance, 
other  areas  require  Soldiers  to  invest  large  portions  of  time  to  attempt  to  resolve  networking 
issues.  Still,  in  other  locations,  the  ability  to  network  is  not  present  and  unlikely  to  change,  thus 
removing  the  possibility  of  interactivity  and  collaboration  between  Soldiers. 

However,  it  is  clear  that  although  NEC  rules  and  regulations  were  seen  as  a  roadblock, 
Soldiers  understood  the  importance  of  restrictions,  with  one  Soldier  stating,  “In  our  situation,  we 
have  a  boatload  of  top  secret  stuff.  We  aren’t  allowed  to  have  WIFI.  Someone  couldn’t  steal  the 
information.  For  our  schoolhouse,  it’s  not  viable.”  A  training  developer  added  to  this  by  stating, 
“we  need  to  have  a  security  structure,”  and  there  needs  to  be  a  “balance  between  desired 
usability  and  network  security.”  Restrictions  are  also  desired  to  discourage  student  misuse,  with 
one  instructor  stating,  “If  we  could  use  a  device  that  had  calls  and  texts  disabled,  it  would  be 
helpful.  It  is  a  temptation  for  students.” 

Yet,  obtaining  this  balance  seems  difficult  according  to  most  Soldiers.  One  instructor 
stated  that  “When  the  students  use  the  tablet,  it  can  be  useful.  But  because  of  various  issues, 
students  don’t  use  them.  Data  at  rest  and  decrypting  are  reasons  they  don’t  use  them.”  Further, 
restrictions  also  impact  application  development,  with  one  training  developer  stating  that 
applications  with  sensitive  data  hinder  the  application  development  process;  “Many  policies 
restrict  access  to  what  is  necessary  to  develop  apps.”  One  training  developer  proposed  forgoing 
security  issues  until  the  end  of  the  development  process  stating  that  the  solution  is  to  “identify 
the  capability  you  want  then  develop  the  app.  Place  the  app  on  a  device  and  test  it  in  a  small 
group  element,  isolated  network,  or  standalone  system  to  allow  refinement  of  the  capability 
according  to  Soldiers’  needs,  contracting  officer’s  representative  (COR)  needs,  etc.  [Then] 
ensure  you  test  and  validate  functionality.  Then,  start  addressing  security  issues  and  network 
requirements. . .”  Thus,  it  is  evident  that  device  related  student  restrictions  are  necessary  to  some 
degree,  but  the  question  becomes,  to  what  extent  do  restrictions  hinder  the  effectiveness  of 
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training,  mobile  device  use,  and  application  development.  These  questions  should  be  considered 
when  making  decisions  related  to  required  and  desired  student  restrictions. 

Related  to  the  aforementioned  restrictions  is  the  issue  of  the  permanency  of  these 
restrictions.  If  restrictions,  such  as  disabling  features  of  the  mobile  device  (e.g.  texting 
capability),  are  important,  one  should  consider  the  pennanency  of  these  restrictions.  Many 
Soldiers  discussed  the  lack  of  permanency  of  restrictions  with  certain  mobile  devices,  such  as  the 
HTC  EVOs  and  other  Android  phones.  These  devices  had  the  ability  to  “self-heal.” 

Specifically,  as  one  training  developer  stated,  “We  could  say  we  don’t  want  these  things  to  work 
and  take  the  strings  out  to  disable  thing  and  when  you  turn  it  off,  it  self-heals.”  Thus,  the 
pennanency  of  restrictions  placed  on  a  device  should  be  known  if  restrictions  are  necessary 

Device,  application,  and  security.  This  subfactor  concerns  the  security 
capabilities/restrictions  of  devices  and  applications.  Here,  the  focus  is  on  the  ability  to  ensure 
the  security  of  both  the  device  and  the  application  and  the  ability  to  circumvent  these  security 
measures.  Regarding  general  device  and  application  security  points  of  consideration  and  issues, 
most  comments  dealt  with  concerns  related  to  the  susceptibility  of  mobile  devices  and 
applications  to  compromises  of  security.  Specifically,  what  is  the  consequence  of  losing  a 
mobile  device  or  losing  data  from  an  application?  One  training  developer  stated,  “I  had  a 
Lieutenant  come  in  and  say  my  tablet  has  been  gone  a  week  and  a  half.”  Although  initial 
security  concerns  did  arise,  the  training  developer  stated  they  were  able  to  “call  our  Verizon  rep 
and  do  our  GPS  and  ping  the  tablet  to  remote  wipe  it  and  turn  it  in.”  Other  concerns  were  related 
to  resourcing  application  delivery,  or  “finding  an  Army  solution  to  the  marketplace.” 

Student  restrictions.  This  subfactor  concerns  the  restrictions  or  limitations  placed  on 
AIT  students  as  part  of  their  training  program.  Concerns  related  to  restrictions  arose  from  issues 
regarding  the  susceptibility  of  hacking  or  circumvention  and  consequences  of  possible  student 
misuse.  Most  of  the  circumvention  concerns  were  related  to  device  capabilities  and  the 
pennanency  of  restrictions.  As  one  training  developer  stated,  students  “change [d]  control 
numbers;  lockout.  I  couldn’t  get  into  the  phone  without  resetting  it.”  Further,  although  certain 
functionalities  were  disabled  on  the  smartphones,  one  training  developer  claimed  that  “you  saw 
innovative  Soldiers  adapt  the  phones  at  night.”  Other  concerns,  beyond  student  circumvention, 
were  expressed  regarding  leaking  of  sensitive  information,  such  as  a  hacker  finding  out  where  a 
Soldier  is  located  via  GPS  or  the  compromise  of  sensitive  data  “floating  through  the  air.”  Thus, 
it  seems  as  the  sensitivity  of  the  content  as  well  as  the  susceptibility  of  hacking  into  the  content 
or  circumventing  device  restrictions  is  an  important  consideration. 

Participants  expressed  concerns  regarding  possible  student  misuse  of  devices.  Soldiers 
stated  that  the  mobile  devices  would  be  a  “distractor.”  Others  expressed  concern  over  possible 
theft  and  subsequent  distribution  of  mobile  devices  for  profit,  with  one  Soldier  concerned  that, 
“you  start  handing  [mobile  devices]  out  to  AIT  and  it  winds  up  in  a  pawn  shop.”  Other  Soldiers 
offered  actual  examples  of  Soldiers  breaking  “into  the  closet  [and]  making  phone  calls,”  and 
circumventing  restrictions  to  “download  radios”  on  the  mobile  devices.  Multiple  solutions  were 
provided  for  these  issues  such  as  not  issuing  the  mobile  devices  “below  E-6s  in  charge  of  the 
AIT  students,”  instilling  a  sense  of  responsibility  into  the  Soldiers  who  are  given  a  mobile 
device,  or  developing  an  “app  counter  to  tell  how  many  times  an  app  has  been  used.”  These 
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comments  suggest  that  student  misuse  is  a  possibility  and  introducing  mobile  devices  to  Soldiers 
should  consider  these  possible  outcomes. 

Device  management.  This  subfactor  concerns  the  management  of  devices  (e.g.,  how 
devices  should  be  distributed,  locked/secured,  stored,  repaired/maintained).  The  central  concepts 
that  participants  discussed  concerning  device  management  were  accountability  and 
responsibility.  One  training  developer  who  had  been  responsible  for  administering  the  devices  in 
a  pilot  said,  “When  I  talk  to  Verizon  they  give  me  a  spreadsheet  to  watch  them.  I  turn  it  into  a 
spreadsheet  first  name  last  name  and  their  outfit.  They  sign  for  the  cost.  The  agreements  they 
are  signing  for  them.  If  it  were  up  to  me  their  S4  would  sign  for  them.  But  as  a  pilot  program, 
we  are  keeping  track  of  everything  so  we  have  the  accountability.  I’ll  track  the  master  list  of  this 
many  went  to  so-and-so  and  then  it  is  their  job.  It  is  a  transfer  of  equipment." 

There  were  mixed  opinions  as  to  who  should  have  the  responsibility  for  devices.  One 
training  developer  suggested  that  devices  could  be  assigned  to  each  unit:  “They  could  have  the 
devices  but  assigned  to  the  unit  so  they  check  them  out  as  needed.  A  group  could  have  one. 

Even  if  for  each  individual  they  would  sign  them  out  as  needed.  Then  the  units  would  keep  them 
under  lock  and  key  and  charged.  They’d  say  Thursday  we  are  going  to  use  them  and  have  them 
ready.”  One  training  developer  warned  against  taking  time  away  from  leaders  or  instructors  by 
making  them  responsible  for  devices:  “Checking  mobile  devices  takes  more  time  away  from  drill 
sergeants.  Plus  Soldiers  will  still  break  in  and  take  the  phones.”  Another  training  developer 
recalled  a  time  that  responsibility  for  devices  was  given  to  a  leader  and  there  was  a  problem  with 
misuse:  “The  kids  broke  into  the  closet  making  phone  calls  to  boyfriend,  mom.  They  were  kept 
in  the  barracks  and  that  was  the  only  time  that  we  signed  things  over  to  the  S4.  They  saw  where 
he  kept  them.  It  was  a  one-time  incident.  If  you  keep  the  weapons  in  a  security  locker,  why 
weren’t  the  EVOs." 

Device  and  Application  Value 

This  factor  concerns  the  value  associated  with  implementing  digital  applications  and 
mobile  learning.  There  were  a  total  of  56  participant  comments  for  the  Device  and  Application 
Value  factor.  There  was  one  questions  in  the  survey  related  to  the  device  and  application  value 
factor.  Three  subfactors  were  identified  that  helped  classify  the  most  important  considerations 
for  this  factor.  Table  5  displays  the  subfactors  and  the  number  of  participant  responses  for  each 
subfactor.  Each  subfactor  is  discussed  below  with  supporting  evidence  extracted  from  interview 
and  focus  group  data. 

Table  13 


Device  and  Application  Value  Subfactors  Comments 


Subfactor 

Number  of 
Participant  Comments 

Supplement  in-class  training  content 

37 

Self-development  training  content 

8 

Demonstration,  assessment,  testing 

8 
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In  the  survey,  participants  were  asked  to  consider  what  is  important  for  decision  makers 
when  determining  how  to  integrate  digital  applications  and  mobile  technologies  into  AIT. 
Specifically,  participants  were  asked  to  identify  the  importance  of  the  mix  of  instructor  led,  self- 
directed,  and  distributed  learning.  92.2%  of  participants  responded  that  the  mix  of  instructor  led, 
self-directed,  and  distributed  learning  was  either  important  or  very  important.  4%  of  participants 
responded  that  the  mix  of  instructor  led,  self-directed,  and  distributed  learning  was  slightly 
important  or  not  at  all  important. 

Supplement  in-class  training  content.  This  subfactor  concerns  the  use  of  handheld 
mobile  devices  to  supplement  training  and  the  program  of  instruction  for  a  course.  A  vast 
majority  of  participants  believed  that  mobile  learning  is  best  used  to  supplement  training  in  AIT. 
One  participant  said  that  mobile  learning  “needs  to  be  blended.”  A  training  developer  said, 
“Instructors  must  integrate  apps/devices  into  instruction  and  reorganize  how  they  teach  to 
incorporate  the  app.”  One  student  cautioned  that  students  should  be  able  to  use  mobile  devices 
as  aids  but  must  not  become  reliant  on  them  to  do  their  job:  “If  they  rely  on  that  to  leam  how  to 
do  it  and  then  get  out  to  the  range  and  can’t  do  it  without  the  tablet.  For  common  tasks  that  you 
need  to  know  by  heart  you  are  developing  a  crutch.” 

Participants  provided  several  examples  to  describe  how  mobile  learning  can  supplement 
AIT.  Most  of  the  examples  discussed  the  effectiveness  of  mobile  devices  in  early  training  or  as 
practice.  One  student  said,  “Using  it  in  the  crawl  phase  of  crawl-walk-run  would  be  extremely 
useful  in  shortening  the  time  spent  in  the  crawl  phase.”  A  training  developer  said,  “We  used 
materials  for  blended  learning  to  go  along  with  instruction.  Training  aids  for  the  Soldiers.  It 
helps  them  to  get  that  up  front.  It’s  not  intended  to  delete  the  instructor  led  training.”  Multiple 
participants  described  how  students  could  use  devices  to  practice.  One  participant  said, 
“Allowing  them  to  practice  without  the  use  of  the  equipment.  We  don’t  have  enough  equipment 
or  instructors,  but  we  have  enough  computers.”  A  student  said,  “If  you  can  practice  on  a  device 
a  lot  before  you  get  on  the  actual  equipment,  and  you’ve  ran  through  it,  you  are  going  to  perfonn 
a  lot  better  with  the  equipment.  When  I  used  the  equipment  for  the  first  time,  I  already  had  a 
good  idea  of  what  to  do.” 

As  part  of  supplementing  training,  participants  described  the  value  of  having  mobile 
devices  as  a  quick  reference.  One  training  developer  said  that  it  was  useful  to  have  mobile 
devices  as  “references  for  the  manuals,  if  we  don’t  have  printed  copies.”  One  training  developer 
described  how  a  student  would  use  it  as  a  reference:  “If  my  instructor  is  telling  me  to  do  this  you 
forget  the  steps,  well  then  you  can  look  back  real  quick  to  a  reference.  If  they  can  go  back  and 
look,  not  during  a  test,  but  on  a  task,  to  look  at  checklists,  not  necessarily  homework,  but  if  they 
are  in  the  field  and  told  to  change  a  part  out  and  they  don’t  remember  all  the  steps  they  can  bring 
the  app  up  and  it  will  tell  them  the  order  and  complete  the  task  without  going  back  to  the 
instructor.”  Additionally,  it  was  suggested  that  links  to  doctrine  would  help  teach  doctrine 
during  training:  “Having  Army  Regulations  (ARs)  so  people  can  look  them  up  would  be  nice. 
You  can  look  them  up  on  a  computer.  We  don’t  have  a  lot  of  time/access  to  that  information.  I 
can  try  to  look  it  up  on  my  phone,  but  if  the  regulations  were  easily  available,  you  are  thinking  of 
something,  you  would  get  more  of  an  AR  understanding  and  knowledge  of  what  is  actually  said. 
People  will  be  able  to  recall  that  faster.” 
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Replace  training  content.  There  were  several  reservations  about  to  what  extent  mobile 
applications  can  be  used  to  replace  training.  One  training  developer  said,  “A  whole  course 
shouldn’t  be  an  application.”  One  instructor  said,  “You  can’t  beat  equipment  hands-on  training. 
You  can’t  replace  hands-on  equipment  with  an  app.  There  is  a  difference  between  tightening  a 
screw  with  a  screw  driver  and  pressing  a  button  that  simulates  a  screw  going  in.  Hands-on 
training  is  where  it  cannot  be  replaced.”  However,  previous  hard  copy  training  aids  that  had 
been  used  to  supplement  training  can  be  delivered  via  mobile  devices.  One  training  developer 
said  there  was  “lots  of  call  for  hip-pocket  training  and  aids.  Mobile  apps  can  replace  them.” 

Self-development  training  content.  This  subfactor  concerns  the  use  of  handheld  mobile 
devices  to  be  used  for  self-development  training.  Self-development  training  is  training  initiated 
and  motivated  by  the  individual  taking  the  training.  Several  participants  suggested  that  the 
mobile  devices  would  be  useful  for  delivering  self-development  training.  One  training  developer 
said,  “Our  culture  is  a  lot  more  motivated  these  days  to  leam  on  their  own.  They  do  a  lot  more 
research.  It’s  all  just  expected.”  Another  training  developer  said  that  when  the  curriculum  was 
put  on  iPods,  “Soldiers  began  ‘playing’  the  interactive  portions  on  their  own.”  One  training 
developer  described  how  self-development  took  place  when  using  mobile  devices:  “With  mobile 
learning  there  is  the  intent  and  how  they  ended  up  using  it.  Because  of  this  and  that  they  were 
collaborating  after  hours.  Given  tools  that  Soldiers  already  know  how  to  use  they  will  use  them 
to  their  full  capabilities.  It  is  the  conversation  that  is  going  on  in  the  hallway.  It  is  ongoing 
learning  and  that  is  mobile  learning.” 

Participants  suggested  that  mobile  devices  were  effective  for  self-directed  learning  and  a 
valuable  part  of  training  during  down  time.  Participants  suggested  that  Soldiers  are  ready  for 
Self-Directed  Learning.  The  devices  were  seen  as  a  means  to  encourage  self-directed  learning. 
One  participant  said,  “I  think  it  has  had  a  positive  effect  on  the  students.  Before,  we’d  see  a  lot 
of  the  students  lolly-gagging  around  during  down  time.  Now,  they  are  using  the  devices  and 
want  to  learn  instead  of  sitting  around  and  doing  nothing.” 

Demonstration,  assessment,  and  testing.  This  subfactor  concerns  the  development  of 
applications  as  a  method  for  demonstrating  (e.g.,  crew  drills),  assessing  (e.g.,  knowledge 
checks),  or  testing  (e.g.,  using  handheld  mobile  devices  to  test  student  training  content 
knowledge  or  skill)  during  learning.  Several  participants  identified  the  value  of  having  mobile 
devices  to  be  used  for  assessment  or  as  a  check  on  learning.  One  student  described  a  way  that 
assessments  might  be  used:  “The  most  benefit  outside  of  FMs,  the  best  would  be  at  home  in 
place  of  instructor.  If  we  had  something  that  we  could  use  as  a  test,  then  you  don’t  have  to  wait 
for  an  instructor  you  can  address  common  issues.  Even  if  it  is  going  through  medical  tasks,  but 
just  evaluate  and  choose  yes/no  it  will  not  teach  you  but  it  will  prep  you  for  the  next  day.”  One 
training  developer  suggested  that  tests  be  coupled  with  entertainments  games:  “Multiple  choice 
tests,  if  you  get  five  correct  in  a  row  you  get  to  play  the  game.” 

Device  and  Application  Costs 

This  factor  concerns  the  application  used  in  training,  its  creation,  its  use,  and  the  potential 
value  gained  by  utilizing  the  application.  There  were  a  total  of  24  participant  comments 
regarding  this  factor.  There  were  no  questions  in  the  survey  related  to  the  Device  and 
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Application  Costs  factor.  Two  subfactors  were  identified  that  helped  classify  the  most  important 
considerations  for  this  factor.  Table  14  lists  the  subfactors  and  the  number  of  participant 
responses  for  each  subfactor.  Each  subfactor  is  discussed  below  with  supporting  evidence 
extracted  from  interview  and  focus  group  data. 

Table  14 


Device  and  Application  Costs  Subfactors  Comments 


Subfactor 

Number  of 
Participant  Comments 

Device  costs 

17 

Application  development  costs 

5 

Device/ Application  maintenance  costs 

5 

Device  costs.  This  subfactor  concerns  the  costs  associated  with  purchasing  a  device  and 
device  accessories  (including  Device  Peripherals  such  as  projectors,  ruggedized  cases,  styluses, 
etc.).  Several  participants  described  costs  associated  with  devices.  These  concerns  were  focused 
on  the  initial  cost  of  the  device,  responsibility  for  the  device,  and  fear  of  costs  associated  with 
breaking  the  device.  First,  the  participants  realized  that  the  initial  cost  for  purchasing  devices 
would  be  a  substantial  investment.  Furthennore,  devices  may  not  be  cost  efficient  for  effectively 
training  every  MOS.  One  commander  said,  “It  is  expensive.  There  is  a  contract  the  Anny  has  to 
sign. .  .it  is  expensive  but  for  [our  MOS],  because  our  graduation  rate  is  already  high,  it  is  hard  to 
argue  that  we  need  the  app  and  mobile  devices  and  that  it  is  doing  some  dramatic  change.  Even 
though  I  think  it  is  good,  it  will  be  hard  to  prove  that  it  is  worth  it.” 

Participants  discussed  the  responsibility  of  having  mobile  devices.  One  instructor  felt 
that  AIT  students  would  not  be  ready  for  the  responsibility:  “At  the  Platoon  Sergeant  level  that 
could  work,  but  not  the  actual  students.  No  students,  out  of  fiscal  responsibility.  You  start 
handing  out  to  AIT  and  it  winds  up  at  a  pawn  shop.  Security  and  fiscal  responsibility  would 
keep  from  issuing  at  that  level.”  Also,  students  themselves  mentioned  that  there  was  a  fear  of 
being  responsible  for  the  devices:  “Initially  my  class  did  not  want  them  because  they  were  afraid 
of  losing,  damaging  the  devices  and  being  responsible  for  paying  for  them.  A  lot  of  them 
weren’t  happy  with  that  because  they  are  not  cheap.  After  they  did  stuff  with  them  they  warmed 
up  to  them  but  they  weren’t  comfortable  if  they  lost  them.” 

Many  participants  discussed  the  cost  associated  with  breaking  the  device.  Participants 
suggested  that  some  field  training  environments  would  be  inappropriate  for  devices  because  of 
the  potential  for  breaking  the  device.  One  instructor  said,  “In  my  AIT  it  would  get  broken  unless 
you  are  dealing  with  mine  detection.  For  combat  engineer  it  would  be  too  many  buttons.”  For 
many  participants,  durability  of  the  devices  was  an  issue  and  was  also  related  to  a  fear  of  having 
to  replace  devices.  However,  a  pilot  test  administrator  described  the  actual  status  of  the  mobile 
devices  following  a  mobile  learning  pilot:  “Out  of  248  [smartphones]  that  we  lent  AIT  for  a  pilot 
[test],  I  had  2  with  water  damage,  9  total  problems,  1  internally  fried,  6  screens  broken.  They 
were  still  under  warranty.”  Perhaps  warranty  or  phone  insurance  against  damage  could  be  used 
to  minimize  the  fear  of  associated  costs  due  to  damage. 
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Application  development  costs.  This  subfactor  concerns  the  costs  associated  with 
developing  training  applications  to  be  used  on  handheld  mobile  devices.  The  development  of 
training  applications  for  mobile  devices  can  be  costly.  Training  developers,  technical  experts, 
and  developer  licenses  represent  the  main  application  development  costs.  One  training 
developer  noted  that  “[the  contracting  process  [is]  not  ideal  for  app  development”  and  that  it  is 
“easier  to  overcome  with  government  workers.”  Another  training  developer  suggested  that 
“[the]  cost  is  offset  by  printing.  Everything  in  the  Army  library  is  pdf.” 

Device/application  maintenance  costs.  This  subfactor  concerns  the  costs  associated 
with  maintaining  and  upgrading  devices  and  applications.  Examples  of  maintenance  and 
upgrades  include,  but  are  not  limited  to,  simple  device  repair  (e.g.,  damaged  screen),  device 
operating  system  (OS)  upgrades,  application  bug  removal,  and  application  upgrades  intended  to 
reflect  changes  in  training  content.  Most  of  the  comments  related  to  this  subfactor  focused  on 
the  need  to  conduct  maintenance  on  applications  and  upgrading  applications.  One  training 
developer  suggested,  “There  needs  to  be  a  feedback  mechanism  for  users  to  provide  feedback  [to 
developers].”  This  type  of  a  mechanism  would  allow  for  users  and  instructors  to  identify 
changes  that  could  improve  applications.  It  was  evident  that  changes  and  upgrades  to 
applications  need  to  factor  in  how  long  it  will  take  to  make  corrections  and  related  costs. 

Additional  subfactors.  Through  analysis  of  the  data  and  comparing  the  results  to  the 
findings  from  the  literature  review,  the  research  team  generated  additional  subfactors  that  were 
not  specifically  identified  by  data  collection  participants.  These  subfactors  are  based  on 
psychological  research  or  instructional  design  practices: 

•  Feedback  requirements:  providing  students  feedback  during  training 

•  Level  of  fidelity  of  training:  the  realism  of  training  compared  to  the  practical 
environment 

•  Aided  or  unaided  training:  the  extent  to  which  training  is  aided  by  an  instructor 

•  Inherent  training  risks  and  consequences  of  performance:  the  extent  to  which 
physical  danger  is  associated  with  the  training 

•  Need  for  peripherals:  the  extent  to  which  peripherals  are  needed  for  use  with 
handheld  devices  for  training 

•  Reusability:  the  aspects  of  mobile  learning  that  are  reusable 

•  Network  costs:  costs  associated  with  establishing  a  network  to  use  with  handheld 
devices. 


Discussion 

Building  on  the  literature  review,  the  quantitative  and  qualitative  results  presented  a 
comprehensive  list  of  the  factors  and  subfactors  most  in  need  of  consideration  to  effectively 
design  and  implement  digital  applications  using  handheld  mobile  devices  in  AIT.  As  described 
in  the  results  section,  the  factors  and  subfactors  provided  a  practical  organization  representing 
the  issues  and  solutions  that  decision  makers  would  need  to  design  and  implement  digital 
applications  using  handheld  mobile  devices  in  AIT.  The  research  team  used  the  data  collection 
results  and  the  lessons  learned  in  the  literature  review  to  create  a  decision  guide. 
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The  data  collection  produced  10  factors  and  35  subfactors.  These  factors  and  subfactors 
were  included  in  the  design  of  the  decision  guide.  Using  the  quotes,  data,  and  literature  review, 
the  research  team  created  issues  and  potential  solutions  to  those  issues  to  help  decision  makers 
design  and  implement  mobile  learning.  In  addition  to  the  35  sub  factors,  7  more  subfactors  were 
added  to  the  decision  guide.  Some  of  these  subfactors  lacked  a  sufficient  amount  of  support  to 
single  out  in  the  results,  but  anecdotal  support  from  interviews  and  focus  groups  combined  with 
lessons  learned  from  the  literature  review  was  sufficient  enough  to  identify  issues  and  solutions 
that  needed  to  be  included  in  the  guide.  The  other  subfactors  that  were  added  to  the  guide,  but 
not  described  in  the  results  section  were  added  because  the  infonnation  learned  in  the  results 
section  led  to  research  team  insight  that  needed  to  be  included  in  the  decision  guide.  Table  15 
reviews  the  subfactors  that  were  added  to  the  Decision  Guide,  but  were  not  described  in  the 
results  section. 


Table  15 

Decision  Guide  Subfactors  not  Included  in  the  Results  Section 


Factor 

Subfactor 

Description 

Training 

Content 

Feedback 

requirements 

Concerns  the  need  to  provide  students  feedback  during 
training. 

Training 

Content 

Level  of  fidelity  of 
training/ assessment 
environment 

Concerns  the  level  of  fidelity  associated  with  the 
training  environment  compared  to  the  execution  of 
learned  tasks  in  the  practical  environment. 

Training 

Content 

Aided/Unaided 

Concerns  whether  training  is  aided  or  unaided. 
Specifically,  is  training  delivery  accompanied  with 
instructor  assistance,  or  do  students  complete  training 
without  the  assistance  of  an  instructor? 

Training 

Content 

Inherent  training 
risks  and 
consequences  of 
perfonnance 

Concerns  whether  there  is  any  physical  danger  or  risk 
associated  with  training  and  the  consequences  of 
committing  an  error  in  training. 

Device 

Need  for 
peripherals 

Concerns  the  need  to  have  additional  peripherals  (e.g., 
projector)  to  be  used  in  conjunction  with  the  devices 
to  deliver  training. 

Device  and 
Application 
Value 

Reusability 

Concerns  what  aspects  of  mobile  learning  are 
reusable. 

Device  and 
Application 
Costs 

Network  costs 

Concerns  the  costs  associated  with  using  or 
establishing  a  network  for  use  with  the  handheld 
mobile  devices. 

Following  the  analysis,  the  research  team  created  a  means  for  organizing  the  factors. 
These  meta-factors  were  used  in  the  decision  guide  to  better  organize  information  for  decision 
makers  and  provide  further  understanding  as  to  how  the  factors  are  related.  The  three  factors 
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were  training  methods  and  delivery,  human  and  contextual  factors,  and  hardware  and 
infrastructure  capabilities  and  constraints.  Table  16  lists  the  meta-factors  and  identifies  which 
factors  belong  to  each  meta-factor. 

Table  16 

Metafactors  and  Related  Factors 

Metafactor 

Factor 

Training  methods  and  delivery 

Training  methods  and  delivery 

Training  methods  and  delivery 

Human  and  contextual  factors 

Human  and  contextual  factors 

Human  and  contextual  factors 

Hardware  and  infrastructure  capabilities  and  constraints 

Hardware  and  infrastructure  capabilities  and  constraints 

Application 

Training  Context 

Training  Content 

Students 

Instructors 

Institutional  Support 

Device 

Networking,  Security, 
and  Implementation 

The  purpose  of  the  guide  was  to  assist  in  identifying  important  information  regarding 
how  student,  course,  instruction,  and  learning  environment  issues  affect  decisions  of  where, 
when,  and  how  to  integrate  digital  applications  and  mobile  technologies  within  AIT  courses. 

This  guide  was  designed  to  aid  those  who  have  decided  that  they  want  to  employ  handheld 
mobile  devices  and  learning  applications  in  their  training  programs.  While  the  issues  and 
suggestions  reviewed  could  be  of  some  use  in  initially  considering  these  technologies  as  a 
potential  training  option,  the  decision  to  implement  handheld  mobile  devices  and  programs  was 
left  to  individual  commanders  and  other  key  decision  makers  with  full  knowledge  of  their 
available  resources,  funding,  and  unique  training  requirements  and  environments.  Therefore,  the 
guide  assumed  that  the  decision  to  explore  employing  mobile  learning  technologies  has  largely 
been  made. 

The  target  audience  for  the  guide  included  anyone  interested  in  developing  and 
integrating  digital  applications  and/or  handheld  mobile  technologies  to  support  training, 
including  but  not  limited  to: 

1 .  AIT  Battalion  Commanders 

2.  AIT  Battery  and  Company  Commanders 

3.  Training  and  course  managers 

4.  Instructors,  training  developers,  and  instructional  designers 

The  guide  was  structured  to  aid  the  identification  of  key  issues  and  potential  solutions 
related  to  implementing  handheld  mobile  technologies  and  digital  applications  into  existing  or 
proposed  training  courses.  The  guide  was  also  structured  to  provide  a  logical,  deliberate  process 
and  checklists  for  the  user  to  critically  examine  recommendations  provided  from  published 
research  and  to  detennine  whether  these  issues  and  potential  solutions  apply  to  their  specific 
training  context.  This  information  was  organized  into  a  series  of  meta-factors,  factors,  and 
subfactors  levels  related  to  key  training,  device,  and  application  constructs. 
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The  guide  was  organized  into  two  sections.  The  two  sections  were  designed  to  provide 
different  levels  of  detail,  allowing  users  the  ability  to  consider  more  general  descriptions  of 
issues  and  solutions  at  a  factor  level  and  then  delve  into  more  detail  for  each  of  the  factors  at  the 
subfactor  level.  Section  intended  to  give  a  broad,  upper  level  overview  of  information  and  issues 
to  consider  when  implementing  handheld  mobile  devices  and  digital  applications  that  could  also 
be  used  as  an  initial  critical  consideration  of  key  issues.  Section  2  provided  a  more  detailed 
examination  at  the  more  specific  subfactor  level,  which  provided  significantly  more  information 
related  to  the  impact  of  the  subfactors  on  the  decision  to  implement  a  handheld  mobile 
technology  or  digital  application  into  an  existing  or  proposed  course. 

Following  the  creation  of  the  draft  decision  guide,  several  participants  from  the  data 
collection  were  contacted  to  review  the  guide  and  provide  input  on  the  factors, 
recommendations,  and  guidelines  put  forth  in  the  guide.  The  final  decision  guide  was  structured 
to  provide  not  only  a  framework  for  making  decisions  regarding  the  use  of  handheld  devices  in 
AIT,  but  also  provides  infonnation  to  educate  users  of  the  guide  on  issues  that  may  need  to  be 
considered. 

Limitations 

There  were  limitations  that  had  an  effect  on  this  research  and  the  design  of  the  decision 
guide.  First,  at  the  time  of  this  research,  the  Army’s  view  of  the  use  of  handheld  devices  in 
training  was  still  evolving  and  there  was  no  official  policy  regarding  the  use  of  such  devices  in 
training.  Second,  and  related  to  the  novelty  of  handheld  devices  being  used  in  Anny  training, 
there  were  a  limited  number  of  SMEs  from  whom  to  collect  data.  There  simply  were  not  many 
individuals  with  experience  designing  or  implementing  training  with  handheld  devices.  Third, 
research  concerning  the  use  of  mobile  learning  devices  for  training  and  educational  purposes  was 
in  its  infancy  at  the  time  of  this  research.  Outcomes  on  training  (e.g.,  less  training  time  required, 
greater  retention  of  information,  etc.)  have  not  been  empirically  evaluated  yet. 

All  of  these  limitations  resulted  in  the  decision  guide  being  more  preliminary  in  nature 
than  a  traditional  decision  guide.  For  example,  the  decision  guide  includes  descriptions  of  each 
factor  and  subfactor,  potential  issues  that  may  be  encountered  and  possible  solutions  rather  than 
being  structured  strictly  as  a  decision  tree.  This  level  of  detail  was  intended  to  help  inform  an 
audience  for  whom  much  of  this  information  would  be  new  and  to  therefore  increase  usability 
and  value  of  the  guide. 

Future  Use  and  Dissemination 

The  decision  guide  was  developed  with  an  orientation  towards  providing  AIT 
commanders  and  decision  makers  a  useful  tool  to  help  guide  their  implementation  of  mobile 
devices  into  AIT.  Some  concerns  expressed  by  participants  during  this  project  indicated  that  the 
AIT  trainee  population  may  present  unique  concerns  in  utilizing  mobile  devices  in  AIT.  For 
example,  most  AIT  Soldiers  are  quite  young,  and  while  they  have  more  independence  in  AIT 
that  they  had  in  basic  training,  there  are  still  concerns  about  allowing  AIT  Soldiers  freedom  with 
these  devices  in  their  barracks.  These  concerns  stem  from  a  mistrust  of  AIT  Soldiers  taking  care 
of  these  devices,  managing  their  time  appropriately,  using  the  devices  for  official  purposes  only, 
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and  not  meddling  with  the  settings  and  restrictions  placed  on  them  by  the  trainers  and  technology 
support  staff.  However,  aside  from  these  issues,  there  were  few  concerns  or  lessons  learned  that 
were  uniquely  limited  to  an  AIT  environment.  Many  of  the  concerns  would  be  appropriate  for 
other  Anny  training  environments.  As  such,  this  guide  could  easily  be  useful  for  commanders  of 
other  training  organizations  and  other  decision  makers  considering  how  best  to  implement 
mobile  devices  to  help  achieve  their  training  goals. 
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ACRONYMS 


2LT 

2  nd  Lieutenant 

ADDIE 

AIT 

ALM 

AR 

ARCIC 

ARI 

Analysis,  Design,  Development,  Implementation,  and  Evaluation 
Advanced  Individual  Training 

US  Army  Learning  Model 

Anny  Regulation 

US  Army  Capabilities  Integration  Center 

Anny  Research  Institute 

BLSS 

BOLC 

Blended  Learning  Satisfaction  Scale 

Basic  Officer  Leaders  Course 

CAC 

CAC 

CASS 

CoEs 

COR 

CPT 

CSDA 

Combined  Anns  Center 

Common  Access  Card 

Case  analysis  attitude  scale 

Centers  of  Excellence 

Contracting  Officer’s  Representative 

Captain 

Connecting  Soldiers  to  Digital  Applications 

DCG-IMT 

DTIC 

Deputy  Commanding  General  for  Initial  Military  Training 
Defense  Technical  Information  Center 

FAQ 

FM 

Frequently  Asked  Questions 

Field  Manual 

GPS 

GS 

Global  positioning  system 

General  Schedule  (US  Civil  Service) 

IET 

iOS 

IMI 

ITE 

Initial  Entry  Training 
iPhone  Operating  System 

Interactive  Multimedia  Instruction 

Integrated  Training  Environment 

LTC 

Lieutenant  Colonel 

MAJ 

MDMP 

MOS 

MS 

Major 

Military  Decision  Making  Process 

Military  Occupational  Specialty 

Metacognition  scale 

NCO 

NEC 

Noncommissioned  Officer 

Network  Enterprise  Center 
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OE 

Operational  Environment 

PBL 

Problem-Based  Learning 

PDF 

Adobe  Portable  Document  Format  file 

PFC 

Private  First  Class 

POI 

Program  of  Instruction 

PPT 

Powerpoint 

PV2 

Private  2 

RFID 

Radio  Frequency  Identification 

SBL 

Studio-based  Learning 

SCoE 

Sustainment  Center  of  Excellence 

SDL 

Self-Directed  Learning 

SFC 

Sergeant  First  Class 

SMART 

Specific,  Measurable,  Achievable,  Realistic,  and  Time-based 

SME 

Subject  Matter  Expert/Expertise 

SRL 

Self-Regulated  Learning 

SSG 

Staff  Sergeant 

TADSS 

Training  Aids,  Devices,  Simulators,  and  Simulations 

TM 

Training  Manual 

TRADOC 

U.S.  Anny  Training  and  Doctrine  Command 

78 


Appendix  A:  Data  Collection  Protocol 


Mobile  Learning  Protocol 
(2/20/2012) 


Introduction  &  Research  Purpose 


Good  morning/good  afternoon  and  thank  you  for  taking  the  time  to  participate  in  this  (focus 
group/interview).  My  name  is  ( Facilitator )  and  these  are  my  colleagues,  (Recorder),  (ARI).  (Recorder)  and 
I  work  for  a  company  called  ICF  International  and  (ARI)  is  from  the  US  Army  Research  Institute.  We  are 
all  part  of  a  research  team  that  the  US  Army  Research  Institute  (ARI)  has  put  together  to  develop  a 
practical  decision  guide  for  key  decision  makers  to  use  in  determining  the  utility  of  digital  applications  and 
mobile  technologies  for  AIT  and  to  develop  metrics  for  assessing  their  impact  on  training.  As  part  of  this 
endeavor,  we  are  interviewing  subject  matter  experts  (SMEs),  such  as  yourself,  who  have  participated  in 
the  Connecting  Soldiers  to  Digital  Applications  (CSDA)  pilot  tests  or  have  past  experience  in  this  area. 
Our  goal  is  to  identify  lessons  learned,  best  practices,  and  critical  decision  making  keys  for  incorporating 
this  technology  into  AIT. 


The  (interview/focus  group)  session  will  take  (60/90)  minutes  to  complete. 


Privacy  Act  Statement  &  Consent  Form 


Please  note  that  your  participation  is  voluntary  -  there  are  no  consequences  if  you  choose  not  to 
participate.  Everything  you  say  will  remain  confidential.  We  will  be  transcribing  your  responses  on  a 
laptop,  but  our  analysis  and  reporting  of  your  responses  will  be  at  the  group  or  aggregate  level — not  at 
the  individual  level.  No  single  individual’s  comments  will  be  referenced  in  any  report  or  presentation  in  a 
manner  that  could  potentially  identify  the  specific  source,  so  please  be  as  candid  as  possible  in  your 
responses  to  our  questions. 


To  more  fully  explain  the  confidentiality  process  and  how  we  will  be  using  the  information  you  provide 
today,  I  have  a  privacy  statement  and  consent  form  for  you  to  read  over.  Please  take  a  few  minutes  to 
read  over  both  documents.  If  you  choose  to  participate,  please  sign  the  second  page  of  the  consent  form 
and  indicate  that  you  are  over  18  years  old  and  are  voluntarily  agreeing  to  participate  (Have  them  sign  the 
digital  recorder  block  as  well  if  applicable)  Please  let  me  know  if  you  have  any  questions  about  the 
privacy  statement,  consent  form,  or  the  session  today.  If  you  are  not  18  yrs  of  age  or  choose  not  to 
participate,  please  return  your  forms  to  me  and  return  to  your  normal  duties.  (Wait,  until  it  looks  as  though 
everyone  has  signed  the  forms  and  then  ask  for  the  signed  page  of  the  consent  forms).  Once  you  have 
signed  your  form,  please  return  the  signed  page  to  me.  You  may  keep  the  remainder  of  the  document 
and  the  summary  provided  by  the  privacy  act  statement  for  your  records  and  later  reference. 
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(Focus  Groups  only) 

Since  your  comments  and  suggestions  are  critical  to  this  research,  we  are 
maintain  the  group’s  confidentiality.  Can  we  all  agree  that  what  is  said  in  this  room 

asking  all  of  you  to 
stays  in  this  room? 

To  make  our  discussion  go  smoothly,  let’s  lay  out  some  ground  rules: 

•  Please  speak  clearly  and  one  at  a  time 

•  Please  avoid  sidebar  conversations 

•  There  are  no  right  or  wrong  answers 

•  We  want  to  hear  the  good  and  the  bad 

•  Any  and  all  differences  of  opinion  should  be  respected  and  valued 

Do  you  have  any  questions  for  me  at  this  time  either  in  terms  of  the  content  of  our  conversation  or  this 
research  effort?  (Answer  any  questions  that  may  arise). 

[Background  Worksheet  &  Questionnaire  | 

Before  we  begin  our  discussion,  we  have  a  background  worksheet  and  a  brief  questionnaire  for  you  to  fill 
out. 
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Background  Worksheet 
(Please  print  all  responses  clearly) 


1.  Rank: _ 

2.  Position  (circle  one): 

Student  Instructor  Training  Developer  CDR  1SG 

Other  (please  identify): _ 

3.  Have  you  been  involved  in  any  effort  to  use  digital  applications  and  mobile  technology  during 
training?  Yes  /  No  (Circle  one).  If  “Yes”,  briefly  describe  the  effort  and  how  you  were  involved. 
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Supplement  Worksheet 

(Only  for  those  consenting  individuals  unable  to  participate  in  the  interview/focus 

group  session) 


1 .  What  previous  experience  have  you  had  with  integrating  digital  applications  and  mobile 
technologies  into  training?  ( Mark  all  that  apply). 


□  Evaluated  the  effectiveness  of  digital  applications  and  /  or  mobile  technologies  for  training 

□  Developed  digital  applications  and  /  or  mobile  technologies  for  training  purposes 

□  Used  digital  applications  and  /  or  mobile  technologies  when  conducting  training 

^  Been  a  student  in  a  class  or  received  training  where  digital  applications  and  /  or  mobile 
technologies  were  used 

^  Conducted  classes  on  how  to  design,  develop,  and/or  maintain  digital  apps  for  military 
settings 

^  OTHER  (please  describe  in  the  space  below  any  additional  experiences  (not  listed  above) 
you  have  had  with  digital  applications  and  mobile  technologies  in  training  settings): 


2.  Briefly  identify  (in  bullet  form;  label  as  indicated)  any  lessons  learned  (LL),  or 

recommendations  (R)  that  would  be  helpful  when  making  the  decision  to  integrate  digital 
applications  and  mobile  technologies  into  AIT  training.  For  example,  you  could  identify: 

What  were  some  of  the  factors  that  made  using  digital  applications  and  mobile 
technologies  in  AIT  effective? 

What  were  some  of  the  roadblock  or  obstacles  you  encounter  when  using  digital 
applications  and  mobile  technologies  in  AIT? 
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What  methods  or  techniques  did  you  use  to  overcome  roadblocks  or  obstacles 
encountered  when  using  digital  applications  and  mobile  technologies  in  AIT? 
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Mobile  Learning  Questionnaire 

1 .  Current  publications  have  identified  a  number  of  instructional  practices  commonly  associated  with  using  digital  applications  and  mobile 
technologies  in  training  environments.  Please  rate  your  experience  to  date  using  digital  applications  and  mobile  technologies  with  the  identified 
instructional  practices.  Please  use  the  following  scale  to  rate  your  level  of  experience  for  each  identified  practice: 

Experience  Scale 

1  -  I  have  experience  using  digital  application  and  mobile  technologies  with  this  instructional  practice. 

2  -  I  have  experience  with  this  instructional  practice,  but  no  specific  experience  using  digital  applications  and  mobile  technologies  in 
combination  with  this  instructional  practice. 

3  -  I  have  very  little  or  no  experience  using  this  instructional  practice  either  alone  or  in  conjunction  with  mobile  devices/technologies 


INSTRUCTIONS:  Please  select  the  response  that  best  reflects  your  experience  with  using  digital  applications/mobile 
technologies  with  the  instructional  practice  identified  below. _ _ 


Instructional  Practice 

Experience 

Rating 

1. 

Encouraging  Self-Directed  Learning  -  Encouraging  students  to  address  their  own  learning  needs  outside  of 
directed  course  or  training  requirements.  Example:  Providing  an  online  forum  for  students  to  share 
developmental  resources  for  their  own  self-development. 

1  2 

3 

2. 

Using  Goal  Setting  -  Having  students  identify  performance  or  learning  goals  and  plan  for  achieving  them. 
Example:  Using  a  skill-based  goal  setting  application  to  develop  learning  goals  and  monitor  progress  towards 
achieving  them. 

1  2 

3 

3. 

Using  Problem  or  Scenario  Based  Learning  -  Training  through  the  use  of  complex,  scenario  based  problems. 
Example:  Requiring  students  to  solve  an  equipment  malfunction  problem  that  can  be  examined  and  tested 
using  apps  on  a  smart  phone. 

1  2 

3 

4. 

Using  Scaffolding  or  Instructor  Support  -  Instructors,  peers,  or  others  providing  support  and  structure  that 
allows  students  to  reach  higher/new  levels  of  understanding  learning.  Example:  Using  smartphones  apps  to 
provide  learning  hints  and  feedback  based  on  specific  student  responses/performance 

1  2 

3 

5. 

Using  Experiential  Learning  -  Encouraging  students  to  learn  through  their  own  direct  experiences  to  develop 
required  skills  and  knowledge.  Example:  Requiring  students  to  work  through  a  training  app  that  requires  them 
to  review  the  basic  operation  and  functions  of  a  piece  of  equipment  before  seeing  it  in  the  classroom. 

1  2 

3 

6. 

Using  Collaborative  and  Cooperative  Learning-  Encouraging  students  to  work  with  other  students  to 
accomplish  shared  goals.  Example:  Having  students  participate  in  an  online  student  forum  in  which  teams  solve 
problems. 

1  2 

3 

7. 

Using  Situational  Learning-  Delivering  training  within  the  same  or  very  similar  context  in  which  it  is  expected  to 
be  applied.  Example:  Providing  a  3-dimensional  representation  of  equipment  on  a  smart  phone  that  can  be 
consulted  when  training  day-to-day  equipment  use. 

1  2 

3 
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2.  INSTRUCTIONS:  On  the  following  page,  for  each  instructional  practice  identified  please  indicate  if,  based  on  your  experiences,  you  believe 
using  digital  applications  and  mobile  technologies  will  have  a  positive,  a  negative  or  no  effect  on  each  of  the  listed  training  outcomes  in  an  AIT 
environment .  Please  use  the  impact  scale  provided  below  to  rate  the  impact  on  each  outcome.  If  you  do  not  have  an  opinion  or  have  no  experience  with 
the  described  instructional  practice,  please  indicate  N/O  (No  Opinion).  Provide  a  rating  for  each  instructional  practice  as  it  relates  to  each  of  the  training 
outcomes. 

Impact  Scale 

1  =  Positive  Impact 

2  =  No  Impact 

3  =  Negative  Impact 

N/O  =  No  Opinion. 

One  way  of  completing  these  ratings  is  to  ask  yourself: 

“At  AIT,  the  use  of  digital  applications  and  mobile  technologies  with  ( insert  Instructional  Practice)  will  have  a  ( insert  impact  rating)  on  ( insert 
training  outcome).” 


For  EXAMPLE,  one’s  completed  ratings  for  “Encouraging  Self-directed  Learning”  might  look  like  the  following: 


INSTRUCTIONAL  PRACTICES  (using  digital 
applications  and  mobile  technologies) 

Increase  student 
motivation  to 
learn 

Increase 
retention  of 
learned  skills 
and  knowledge 
throughout  AIT 

Enhance 

Student 

performance 

(meeting  or 

exceeding 

standards) 

Increase  transfer 
of  knowledge 
and  skills  to 
operational 
contexts 

Make  training 
more  efficient 
(reducing  time 
spent  or  costs  of 
training) 

1.  Encouraging  Self-Directed  Learning  -  Encouraging 
students  to  address  their  own  learning  needs  outside  of 
directed  course  or  training  requirements.  Example: 
Providing  an  online  forum  for  students  to  share 
developmental  resources  for  their  own  self¬ 
development. 

2 

3 

1 

N/O 

2 
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Remember  to  use  the  following  method  of  thinking  when  rating  the  impact  of  using  digital  applications 

Scale: 

and  mobile  technologies  (with  each  instructional  practices)  on  each  of  the  different  training  outcomes: 

1=Positive  Effect 

2=No  Effect 

“At  AIT,  the  use  of  diqital  applications  and  mobile  technologies  with  (insert  Instructional  Practice)  will  have  a 

( insert  impact  rating)  on  ( insert  training  outcome).” 

3=  Negative  Effect 

N/O  =  No  opinion 

INSTRUCTIONAL  PRACTICES  (using  digital 
applications  and  mobile  technologies) 

TRAINING  OUTCOMES 

Increase 

student 

motivation  to 

learn 

Increase 

retention  of 

learned  skills 
and  knowledge 
throughout  AIT 

Enhance 

Student 
performance 
(meeting  or 
exceeding 
standards) 

Increase 

transfer  of 
knowledge  and 
skills  to 
operational 

contexts 

Make  training 
more  efficient 
(reducing  time 
spent  or  costs 
of  training) 

1.  Encouraging  Self-Directed  Learning  -  Encouraging 
students  to  address  their  own  learning  needs  outside  of 
directed  course  or  training  requirements.  Example: 
Providing  an  online  forum  for  students  to  share 
developmental  resources  for  their  own  self-development. 

2.  Using  Goal  Setting  -  Having  students  identify 

performance  or  learning  goals  and  plan  for  achieving 
them.  Example:  Using  a  skill-based  goal  setting 
application  to  develop  learning  goals  and  monitor 
progress  towards  achieving  them. 

3.  Using  Problem  or  Scenario  Based  Learning  -  T raining 
through  the  use  of  complex,  scenario  based  problems. 
Example:  Requiring  students  to  solve  an  equipment 
malfunction  problem  that  can  be  examined  and  tested 
using  apps  on  a  smart  phone. 

4.  Using  Scaffolding  or  Instructor  Support  -  Instructors, 
peers,  or  others  providing  support  and  structure  that 
allows  students  to  reach  higher/new  levels  of 
understanding  learning.  Example:  Using  smartphones 
apps  to  provide  learning  hints  and  feedback  based  on 
specific  student  responses/performance 

(Continue  onto  next  page) 
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INSTRUCTIONAL  PRACTICES  (using  digital 
applications  and  mobile  technologies) 

TRAINING  OUTCOMES 

Increase 

student 

motivation  to 

learn 

Increase 

retention  of 

learned  skills 
and  knowledge 
throughout  AIT 

Enhance 

Student 
performance 
(meeting  or 
exceeding 
standards) 

Increase 

transfer  of 
knowledge  and 
skills  to 
operational 
contexts 

Make  training 
more  efficient 
(reducing  time 
spent  or  costs 
of  training) 

5.  Using  Experiential  Learning  -  Encouraging  students  to 
learn  through  their  own  direct  experiences  to  develop 
required  skills  and  knowledge.  Example:  Requiring 
students  to  work  through  a  training  app  that  requires  them 
to  review  the  basic  operation  and  functions  of  a  piece  of 
equipment  before  seeing  it  in  the  classroom. 

6.  Using  Collaborative  and  Cooperative  Learning- 

Encouraging  students  to  work  with  other  students  to 
accomplish  shared  goals.  Example:  Having  students 
participate  in  an  online  student  forum  in  which  teams 
solve  problems. 

7.  Using  Situational  Learning-  Delivering  training  within 
the  same  or  very  similar  context  in  which  it  is  expected  to 
be  applied.  Example:  Providing  a  3-dimensional 
representation  of  equipment  on  a  smart  phone  that  can 
be  consulted  when  training  day-to-day  equipment  use. 

8.  Other  learning  strategy: 
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3.  Based  on  your  experiences  using  or  working  with  digital  applications  and  mobile  technologies  during  AIT,  identify  how  effective  you  think 
they  were  in  each  of  the  following  training  environments?  ( Mark  only  one  box  per  row). 


EFFECTIVENESS 

TRAINING 

ENVIRONMENT 

Very  Ineffective 

Ineffective 

Neither 

Effective  nor  Effective 

Ineffective 

Very  Effective 

No  Opinion 

Classroom  Instruction  in 

AIT 

□ 

□ 

□  □ 

□ 

□ 

Field  Instruction  in  AIT 
(e.g.,  hands  on  training  on 
identified  tasks) 

□ 

□ 

□  □ 

□ 

□ 

Concurrent  Training  (e.g. 
reviewing  first  aid  while 
waiting  for  other  Soldiers 
to  complete  their  firing 
iteration) 

□ 

□ 

□  □ 

□ 

□ 

Informal  Instruction  (e.g., 
reviewing  course  content 
in  the  barracks) 

□ 

□ 

□  □ 

□ 

□ 

A-10 


4.  How  important  are  the  following  factors  to  consider  for  decision  makers  when  determining  how  to  integrate  digital  applications  and  mobile 
technologies  into  AIT? 


IMPORTANCE 

Not  at  all 

Slightly 

Somewhat 

Very 

AREAS  OF  CONSIDERATION 

Important 

Important 

Important 

Important 

Important 

No  Opinion 

Training  Environment 

The  objectives  of  the  training  (e.g.,  training  objectives  versus  learning 
objectives) 

□ 

□ 

□ 

□ 

□ 

□ 

The  knowledge,  skill,  and  psychomotor  requirements  of  the  MOS 
being  trained 

□ 

□ 

□ 

□ 

□ 

□ 

The  background  and  qualifications  of  the  instructor  conducting  the 

training 

□ 

□ 

□ 

□ 

□ 

□ 

The  mix  of  instructor  led,  self-directed,  and  distributed  learning 

□ 

□ 

□ 

□ 

□ 

□ 

The  nature  of  the  learning  opportunities  and  what  the  focal  intent  of 
the  training  is  on  (e.g.,  knowledge,  skill  acquisition,  practice  and  feedback) 

□ 

□ 

□ 

□ 

□ 

□ 

The  requirements  of  the  specific  task  being  trained  (e.g.,  some  tasks 

may  require  the  use  of  a  student’s  hands  making  it  difficult  to  hold  the  device 
and  perform  the  task) 

□ 

□ 

□ 

□ 

□ 

□ 

The  availability  of  training  resources,  especially  operational 
equipment,  ranges,  and  simulators 

□ 

□ 

□ 

□ 

□ 

□ 

The  amount  of  time  available  for  each  training  area 

□ 

□ 

□ 

□ 

□ 

□ 

Leadership  support  for  implementing  mobile  devices  for  training 

□ 

□ 

□ 

□ 

□ 

□ 

Users 

The  instructor’s  knowledge  of  mobile  device  capabilities 

□ 

□ 

□ 

□ 

□ 

□ 

The  instructor’s  ability  to  use  mobile  devices  along  with  instructional 
practices 

□ 

□ 

□ 

□ 

□ 

□ 

(Continue  onto  next  page) 
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AREAS  OF  CONSIDERATION 

IMPORTANCE 

Not  at  all  Slightly  Somewhat  Very  No 

Important  Important  Important  Important  Important  Opinion 

The  student’s  background  and  experience 

□ 

□ 

□ 

□ 

□ 

□ 

The  student’s  level  of  motivation  and  learning  orientation 

□ 

□ 

□ 

□ 

□ 

□ 

The  role  of  other  persons  in  training  (e.g.,  peers) 

□ 

□ 

□ 

□ 

□ 

□ 

The  maturity  level  of  students 

□ 

□ 

□ 

□ 

□ 

□ 

The  ability  for  cadre/instructors  to  control  certain  features  of  the 

device 

□ 

□ 

□ 

□ 

□ 

□ 

Device 

Device  capabilities  such  as  memory,  input/output,  capabilities, 
connectivity 

□ 

□ 

□ 

□ 

□ 

□ 

Usability  of  the  device/application  (e.g.,  user  friendliness,  screen 
size,  ease  to  program/set-up,  etc.) 

□ 

□ 

□ 

□ 

□ 

□ 

Utility  of  the  device/application  (i.e.,  how  effective  is  the 
device/application  for  doing  what  you  want  it  to  do?) 

□ 

□ 

□ 

□ 

□ 

□ 

Durability  of  the  device/platform  being  used  for  the  application 

□ 

□ 

□ 

□ 

□ 

□ 

Network  availability 

□ 

□ 

□ 

□ 

□ 

□ 

The  type  of  device  (e.g.,  tablet  versus  smart  phone) 

□ 

□ 

□ 

□ 

□ 

□ 

Add  any  additional  considerations  below  and  rate  the  criticality  of  each 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 
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Focus  Group/Interview  Questions 


Instructions:  Interviewers  /  Facilitators  should  only  ask  the  optional  question  below  when  they 
have  determined  that  participants  have  not  provided  enough  information  on  question  3  of  the 
background  worksheet.  Otherwise,  they  should  start  with  question  1.  Also,  Interviewers  / 
Facilitators  need  to  be  cognizant  of  the  time  available  for  focus  group  /  interview  questions  after 
participants  have  completed  the  questionnaire.  If  time  is  limited,  questions  in  bold  get  priority 
over  the  bulleted  prompt  questions.  Prompt  questions  should  only  be  used  when  time  permits 
and  then  only  after  asking  a  bolded  question  when  the  interviewer  /  facilitator  feels  more  detail 
regarding  that  topic  is  required. 


EEA1 :  How  can  mobile  devices  be  used  in  AIT  to  add  value  to  training? 


Optional  Question:  What  experiences  have  you  had  with  digital  applications  and  mobile 
technologies  in  your  AIT  program? 

1 .  In  your  experience,  what  are  some  of  the  factors  that  have  made  using  digital 
applications  and  mobile  technologies  in  AIT  effective?  Why  were  they  effective? 

In  your  experience,  what  are  the  primary  roadblocks  to  consider  when  integrating  digital 
applications  and  mobile  technologies  in  AIT?  How  were  they  overcome  (e.g.,  work 
around)? 


EEA2:  Training  Environment 


2.  What  types  of  tasks  or  training  objectives  would  be  most  appropriate  for  training 
with  digital  applications  and  mobile  technologies?  Why? 

What  types  of  tasks  would  not  be  appropriate  for  training  with  digital  applications  and/or 
mobile  technologies?  Why?  What  distinguishes  them  from  other  types  of  tasks? 

What  types  of  presentation  formats  (e.g.,  lecture,  audio,  video,  games,  simulation)  have 
been  most  effective  at  delivering  training  in  your  AIT  using  digital  applications  and  /  or 
mobile  technologies?  Why? 

What  types  of  presentation  formats  are  not  well  suited  for  digital  applications  and  mobile 
technologies?  Why? 


EEA3:  User  Considerations 


3.  What  is  the  best  way  to  ensure  instructors  buy  into  the  use  of  digital  applications 
and  mobile  technologies  in  their  training? 

What  is  the  best  way  to  ensure  others  involved  in  the  training  (e.g.,  AIT  commander,  AIT 
Platoon  Sergeant)  buy  into  the  integration  of  digital  applications  and  mobile 
technologies? 
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How  would  student  characteristics  affect  the  success  of  integrating  digital  applications 
and  mobile  technologies  into  training? 


EEA4:  Device  Considerations 


4.  Which  is  the  most  important  device  consideration  when  integrating  digital 
application  and  mobile  technologies  into  AIT  training:  usability,  utility  or 
capability?  Why? 

What  considerations  should  be  made  regarding  usability  in  the  decision  to  integrate 
digital  applications  and  mobile  technologies  into  training  and  why?  Examples  of  usability 
include  things  like  user  friendliness,  durability,  screen  size,  etc. 

What  device  capabilities  are  /should  be  considered  in  the  decision  to  integrate  digital 
applications  and  mobile  technologies  into  training  and  why?  Examples  of  capabilities 
include  bandwidth,  reception,  battery  life,  and  audio/video  resolution. 

What  considerations  should  be  made  regarding  utility  in  the  decision  to  integrate  digital 
applications  and  mobile  technologies  into  training  and  why?  The  utility  of  a  device  refers 
to  how  effective  is  the  device/application  for  doing  what  you  want  it  to  do? 


Concluding  Comments 


5.  Finally,  regarding  our  goal  to  develop  a  decision  guide  for  implementing  mobile 
devices  into  AIT.  Are  there  any  additional  factors,  lessons  learned  or  guidance 
that  you  would  like  to  share  regarding  what  we’ve  been  talking  about? 


Thank  you  again  for  your  time  and  participation.  Your  comments  have  been  very  helpful. 
Before  we  let  you  go,  we  were  wondering  if  you  would  be  willing  to  assist  us  with  reviewing  the 
near  final  version  of  the  decision  guide  once  it  has  been  developed.  This  would  probably  occur 
_ (If  yes,  get  their  email  address). 
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Appendix  B:  Quantitative  Results  -  Effect  of  Using  Mobile  Applications  with 


Ins 


ructional  Practices 


Increase  student  motivation  to  learn 

The  effect  of  using  digtial 
applications  and  mobile 
technologies  with 

Positive  Impact 

No  Impact 

Negative  Impact 

Count 

Row  N 

% 

Count 

Row  N 

% 

Count 

Row  N 

% 

Encouraging  Self  Directed  Learning  effect 
on 

34 

73.9% 

9 

19.6% 

3 

6.5% 

Using  Goal  Setting  effect  on 

34 

77.3% 

8 

18.2% 

2 

4.5% 

Using  Problem  or  Scenario  Based  Learning 
effect  on 

35 

76.1% 

11 

23.9% 

0 

0.0% 

Using  Scaffolding  or  Instructor  Support 
effect  on 

32 

68.1% 

13 

27.7% 

2 

4.3% 

Using  Experiential  learning  effect  on 

35 

74.5% 

11 

23.4% 

1 

2.1% 

Using  Collaborative  and  Cooperative 

Learning  effect  on 

29 

64.4% 

14 

31.1% 

2 

4.4% 

Using  Situational  Learning  effect  on 

39 

84.8% 

7 

15.2% 

0 

0.0% 

Other  Learning  Strategy  effect  on 

2 

100.0% 

0 

0.0% 

0 

0.0% 

Increase  retention  of  learned  skills  and 
knowledge  throughout  AIT 

The  effect  of  using  digtial 
applications  and  mobile 
technologies  with 

Positive  Impact 

No  Impact 

Negative  Impact 

Count 

Row  N 

% 

Count 

Row  N 

% 

Count 

Row  N 

% 

Encouraging  Self  Directed  Learning  effect 
on 

35 

79.5% 

9 

20.5% 

0 

0.0% 

Using  Goal  Setting  effect  on 

29 

65.9% 

15 

34.1% 

0 

0.0% 

Using  Problem  or  Scenario  Based  Learning 
effect  on 

39 

83.0% 

8 

17.0% 

0 

0.0% 

Using  Scaffolding  or  Instructor  Support 
effect  on 

33 

73.3% 

10 

22.2% 

2 

4.4% 

Using  Experiential  learning  effect  on 

42 

89.4% 

5 

10.6% 

0 

0.0% 

Using  Collaborative  and  Cooperative 

Learning  effect  on 

27 

61.4% 

16 

36.4% 

1 

2.3% 

Using  Situational  Learning  effect  on 

42 

91.3% 

3 

6.5% 

1 

2.2% 

Other  Learning  Strategy  effect  on 

2 

100.0% 

0 

0.0% 

0 

0.0% 
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Enhance  Student  performance  (meeting  or 
exceeding  standards) 

The  effect  of  using  digtial 
applications  and  mobile 
technologies  with 

Positive  Impact 

No  Impact 

Negative  Impact 

Count 

Row  N 

% 

Count 

Row  N 

% 

Count 

Row  N 

% 

Encouraging  Self  Directed  Learning  effect 
on 

33 

73.3% 

11 

24.4% 

1 

2.2% 

Using  Goal  Setting  effect  on 

33 

73.3% 

12 

26.7% 

0 

0.0% 

Using  Problem  or  Scenario  Based  Learning 
effect  on 

39 

86.7% 

6 

13.3% 

0 

0.0% 

Using  Scaffolding  or  Instructor  Support 
effect  on 

35 

76.1% 

10 

21.7% 

1 

2.2% 

Using  Experiential  learning  effect  on 

39 

84.8% 

6 

13.0% 

1 

2.2% 

Using  Collaborative  and  Cooperative 

Learning  effect  on 

28 

65.1% 

14 

32.6% 

1 

2.3% 

Using  Situational  Learning  effect  on 

43 

93.5% 

2 

4.3% 

1 

2.2% 

Other  Learning  Strategy  effect  on 

2 

100.0% 

0 

0.0% 

0 

0.0% 

Increase  transfer  of  knowledge  and  skills  to 
operational  contexts 

The  effect  of  using  digtial 
applications  and  mobile 
technologies  with 

Positive  Impact 

No  Impact 

Negative  Impact 

Count 

Row  N 

% 

Count 

Row  N 

% 

Count 

Row  N 

% 

Encouraging  Self  Directed  Learning  effect 
on 

33 

73.3% 

10 

22.2% 

2 

4.4% 

Using  Goal  Setting  effect  on 

25 

59.5% 

17 

40.5% 

0 

0.0% 

Using  Problem  or  Scenario  Based  Learning 
effect  on 

37 

80.4% 

9 

19.6% 

0 

0.0% 

Using  Scaffolding  or  Instructor  Support 
effect  on 

29 

65.9% 

14 

31.8% 

1 

2.3% 

Using  Experiential  learning  effect  on 

41 

87.2% 

6 

12.8% 

0 

0.0% 

Using  Collaborative  and  Cooperative 

Learning  effect  on 

29 

65.9% 

13 

29.5% 

2 

4.5% 

Using  Situational  Learning  effect  on 

40 

87.0% 

6 

13.0% 

0 

0.0% 

Other  Learning  Strategy  effect  on 

2 

100.0% 

0 

0.0% 

0 

0.0% 
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Make  training  more  efficient  (reducing  time 
spent  or  costs  of  training) 

The  effect  of  using  digtial 
applications  and  mobile 
technologies  with 

Positive  Impact 

No  Impact 

Negative  Impact 

Count 

Row  N 

% 

Count 

Row  N 

% 

Count 

Row  N 

% 

Encouraging  Self  Directed  Learning  effect 
on 

28 

63.6% 

11 

25.0% 

5 

1 1 .4% 

Using  Goal  Setting  effect  on 

29 

65.9% 

14 

31.8% 

1 

2.3% 

Using  Problem  or  Scenario  Based  Learning 
effect  on 

32 

71.1% 

8 

17.8% 

5 

11.1% 

Using  Scaffolding  or  Instructor  Support 
effect  on 

28 

63.6% 

11 

25.0% 

5 

1 1 .4% 

Using  Experiential  learning  effect  on 

33 

76.7% 

10 

23.3% 

0 

0.0% 

Using  Collaborative  and  Cooperative 

Learning  effect  on 

25 

56.8% 

14 

31.8% 

5 

1 1 .4% 

Using  Situational  Learning  effect  on 

37 

82.2% 

6 

13.3% 

2 

4.4% 

Other  Learning  Strategy  effect  on 

2 

100.0% 

0 

0.0% 

0 

0.0% 

B-3 


B-4 


Appendix  C:  Decision  Guide 


c-i 


C-2 


A  Practical  Decision  Guide  for 
Integrating  Digital  Applications  and 
Mobile  Technologies  into 
Advanced  Individual  Training 


U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences 
Fort  Belvoir,  Virginia 


Approved  for  public  release;  distribution  is  unlimited. 


A  Practical  Decision  Guide  for  Integrating 
Mobile  Technologies  and  Digital  Applications  in  AIT 


Table  of  Contents 


Introduction . 3 

Practical  Decision  Guide  Background . 3 

Purpose  of  the  Practical  Decision  Guide . 3 

Organization  of  the  Guide . 4 

Target  Audience  for  the  Guide . 5 

Use  of  the  Guide . 5 

Section  1:  Metafactors  and  Factors  Level  . 6 

Metafactor:  Training  Methods  and  Delivery . 7 

Factor:  Application . 7 

Factor:  Training  Context . 8 

Factor:  Training  Content . 9 

Metafactor:  Human  and  Contextual  Factors . 10 

Factor:  Students . 10 

Factor:  Instructors . 11 

Factor:  Institutional  Support . 12 

Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints . 13 

Factor:  Device . 13 

Factor:  Networking,  Security,  and  Implementation  Issues . 14 

Section  2:  Metafactors,  Factors  and  Subfactors  Worksheets . 15 

Metafactor:  Training  Methods  and  Delivery . 16 

Factor:  Application . 16 

Factor:  Training  Context . 24 

Factor:  Training  Content . 27 

Metafactor:  Human  and  Contextual  Factors . 36 

Factor:  Students . 36 

Factor:  Instructors . 37 

Factor:  Institutional  Support . 40 

Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints . 42 

Factor:  Device . 42 

Factor:  Networking  and  Security . 48 

Metafactor:  Overarching  Factors . 53 

Factor:  Training  Value . 53 

Factor:  Training  Cost . 56 

References . 57 

Glossary . 58 


Page  2  of  59 


A  Practical  Decision  Guide  for  Integrating 
Mobile  Technologies  and  Digital  Applications  in  AIT 


Introduction 


Practical  Decision  Guide  Background 

The  U.S.  Army  continually  seeks  innovative  means  to  improve  training,  particularly  in  the  face  of  rapid 
technological  advances  and  increasingly  challenging  Soldier  training  requirements.  A  relatively  new 
area  of  focus  seeks  to  exploit  emerging  interactive,  handheld  mobile  technologies  and  digital 
applications  to  enhance  the  effectiveness  and  efficiency  of  Soldier  training.  For  example,  the 
Connecting  Soldiers  to  Digital  Applications  (CSDA)  initiative  sponsored  and  encouraged  units  to 
examine  the  utility  of  employing  mobile,  handheld  technologies,  and  gaming  solutions  to  bridge  the 
gap  between  Soldier  learning  and  training  as  a  function  of  time,  resources,  and  instructor  contact 
ratios. 

The  United  States  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  working  in 
collaboration  with  ICF  International,  developed  the  following  practical  decision  guide  for  AIT 
commanders  and  training  developers  considering  the  development  and  integration  of  digital 
applications  and  handheld  technologies  in  their  training  programs.  This  guide  is  based  on  an  extensive 
review  of  published  research,  published  reviews  of  training  practices  using  these  technologies,  and 
field  data  collected  from  CSDA  participants  and  training  managers.  These  sources  were  used  to  identify 
the  key  factors  impacting  this  decision  process  and  some  methods  for  resolving  challenges  to  device 
and  application  integration.  While  this  guide  focuses  on  the  integration  and  presentation  of  content  in 
AIT,  its  recommendations  can  readily  be  applied  to  other  training  applications,  contexts,  and  objectives 
as  well.  It  is  also  important  to  keep  in  mind  that  while  these  recommendations  were  made  for 
unclassified  content,  they  can  also  be  applied  to  classified  training  environments  provided  all 
appropriate  security  measures  regarding  content  control,  security  access,  network  integration,  data 
storage,  transmission  and  sharing,  are  carefully  followed. 

Purpose  of  the  Practical  Decision  Guide 

This  guide  was  designed  to  aid  those  who  have  decided  that  they  want  to  employ  handheld  mobile 
devices  and  learning  applications  in  their  training  programs.  While  the  issues  and  suggestions  reviewed 
in  this  guide  could  be  of  some  use  in  initially  considering  these  technologies  as  a  potential  training 
option,  the  decision  to  implement  handheld  mobile  devices  and  programs  is  best  left  to  individual 
commanders  and  other  key  decision  makers  with  full  knowledge  of  their  available  resources,  funding, 
and  unique  training  requirements  and  environments.  Therefore,  this  guide  assumes  that  the  decision 
to  explore  employing  mobile  learning  technologies  has  largely  been  made.  Thus,  its  focus  is  on  how  to 
design,  implement,  and  execute  mobile  learning  options. 

The  purpose  of  this  guide  is  to  assist  in  identifying  important  information  regarding  how  student, 
course,  instruction,  and  learning  environment  issues  affect  decisions  of  where,  when,  and  how  to 
integrate  digital  applications  and  mobile  technologies  within  AIT  courses.  This  information  has  been 
organized  into  a  series  of  Meta  factors.  Factors,  and  Subfactors  levels  related  to  key  training,  device, 
and  application  constructs.  Metafactors  reflect  overarching  constructs  related  to  Training  Value  and 
Cost,  Training  Methods  and  Delivery  Options,  and  Fluman  and  Contextual  Factors.  Table  1  provides  an 
overview  of  each  of  the  Metafactors  and  their  associated  Factors,  and  Subfactors.  A  glossary  of  terms 
is  also  provided  at  the  end  of  this  guide.  Terms  highlighted  in  red  can  be  reviewed  in  the  glossary  by 
clicking  on  them. 
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Table  1.  Decision  Metafactors,  Factors,  and  Subfactors. 


Metafactors 

Factors 

Subfactors 

Application 

Development;  Implementation; 
Information  representation;  User 
Interaction;  Usability;  Utility-Value; 
Compatibility 

Training  Methods  and  Delivery 

Training  Context 

Classroom;  Field/Range;  During 

Personal  Time;  Time  required  or 
duration  or  training; 

Training  Content 

Objectives;  Content  complexity; 

Content  constancy/stability; 
Individual/Collective  task;  Resource 
Requirements;  Training  Feedback; 

Level  of  fidelity  of  training/assessment 
environment;  Aided  vs.  unaided 
instruction;  Inherent  training  risks  and 
consequences  of  performance 

Students 

Attitudes/experiences  with 
devices/applications  and  readiness  for 
self-directed  learning 

Human  and  Contextual  Factors 

Instructors 

Experience/attitudes  toward  device; 
Perceived  benefits  to  instructor 

Institutional  Support 

Broad-based  support  in  Army  culture: 
Support  from  Chain  of  Command 

Hardware  and  Infrastructure 
Capabilities  and  Constraints 

Device 

Utilizing  Device  capabilities;  Usability; 
Durability;  Portability;  Supportability; 
Need  for  Peripherals 

Networking,  Security,  and 

Implementation  Issues 

Networking  capabilities,  restrictions, 
and  policy  requirements;  Device, 
application,  and  data  security;  Student 
restrictions;  Device  management 

Overarching  Factors 

Device  and  Application  Value 

Supplement  in-class  training  content; 
Self-development  training  content; 
Demonstration,  assessment,  testing; 
Reusability 

Device  and  Application  Costs 

Device  costs;  Network  costs; 

Application  development  costs; 
Device/application  maintenance  costs 

Organization  of  the  Guide 

This  document  is  organized  into  two  sections  and  a  glossary  of  terms.  The  two  sections  were  designed 
to  provide  different  levels  of  detail,  allowing  users  the  ability  to  consider  more  general  descriptions  of 
issues  and  solutions  at  a  Factor  level  and  then  delve  into  more  detail  for  each  of  the  Factors  at  the 
Subfactor  level.  Section  1  provides  an  upper  level  examination  of  issues  and  solutions  at  the  Factor 
level.  This  section  is  intended  to  give  a  broad  overview  of  information  and  issues  to  consider  when 
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implementing  handheld  mobile  devices  and  digital  applications  that  could  also  be  used  as  an  initial 
critical  consideration  of  key  issues.  Should  you  want  to  know  more  about  a  particular  factor  and  its  key 
issues  there  are  hyperlinks  that  will  provide  you  with  more  information  in  Section  2.  Section  2  provides 
a  more  detailed  examination  at  the  more  specific  Subfactor  level,  which  provides  significantly  more 
information  related  to  the  impact  of  the  subfactors  on  the  decision  to  implement  a  handheld  mobile 
technology  or  digital  application  into  an  existing  or  proposed  course. 

Target  Audience  for  the  Guide 

The  target  audience  for  this  guide  includes  anyone  interested  in  developing  and  integrating  digital 
applications  and/or  handheld  mobile  technologies  to  support  training,  including  but  not  limited  to: 


1.  AIT  Battalion  Commanders 

2.  AIT  Battery  and  Company  Commanders 

3.  Training  and  course  managers 

4.  Instructors,  training  developers,  and  instructional  designers. 


Use  of  the  Guide 

This  guide  is  structured  to  aid  the  identification  of  key  issues  and  potential  solutions  related  to 
implementing  handheld  mobile  technologies  and  digital  applications  into  existing  or  proposed  training 
courses.  The  guide  is  also  structured  to  provide  a  logical,  deliberate  process  and  checklists  for  the  user 
to  critically  examine  recommendations  provided  from  published  research  and  to  determine  whether 
these  issues  and  potential  solutions  apply  to  their  specific  training  context.  Users  are  encouraged  to 
solicit  input  from  others  regarding  how  relevant  these  issues  and  potential  solutions  are  to  their  unique 
training  environments  and  constraints. 

Use  of  this  guide  should  assist  decision  makers  in  estimating  and  examining: 


1.  Potential  training  value  and  costs 

2.  Training  application,  context,  and  content  considerations 

3.  Student,  Instructor,  and  Institutional  support  considerations 

4.  Device,  Networking,  Security,  and  Implementation  considerations. 


Throughout  the  guide,  each  factor  (Section  1)  and  subfactor  (Section  2)  are  organized  into  an  issues 
section  and  a  solutions  section.  Where  specific  issues  directly  correspond  to  specific  solutions,  the 
suggested  solutions  are  numbered  corresponding  to  the  number  of  the  issue  to  make  it  easier  to  link 
solutions  with  specific  issues.  Issues  and  solutions  that  are  more  general  in  nature  are  bulleted  or 
notated  in  the  "considerations"  section. 

The  guide  can  be  used  in  printed  and  electronic  formats.  Hyperlinks  are  included  in  the  guide  for  ease 
of  locating  relevant  sections  when  viewing  in  electronic  format. 
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Section  1:  Metafactors  and  Factors  Level 


This  section  provides  a  high  level  examination  of  the  following  considerations  and  possible  methods  of 
resolving  any  challenges.  This  information  is  presented  at  the  Metafactor  and  Factor  level. 

1.  Potential  training  value  and  costs 

2.  Training  application,  context,  and  content  considerations 

3.  Student,  Instructor,  and  Institutional  support  considerations 

4.  Device,  Networking,  Security,  and  Implementation  considerations. 

Instructions:  Use  the  following  pages  to  determine  if  the  identified  issues  and  potential  solutions  are 
relevant  at  the  Metafactors  and  Factors  level.  For  more  detail  about  the  issues  described  in  the  factors, 
click  on  the  hyperlinked  term  to  explore  related  Subfactors  within  this  guide. 
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Metafactors:  Training  Methods  and  Delivery 
Factor:  Application 


Description:  This  factor  focuses  on  the  training  application  being  considered  for  employment,  its 
creation,  its  use,  and  the  potential  value  gained  by  utilizing  the  application. 


Issues:  Issues  to  be  considered  regarding  the  application  include: 

(1)  Development:  This  is  a  complex  process  that  can  be  negatively  impacted  by  a  lack 
of  technically  skilled  developers  and/or  resources,  an  unclear  development  process, 
or  an  inconsistency  with  Army  standards. 

(2)  Implementation:  Challenges  arise  in  implementing  the  application  when  instructors 
do  not  effectively  demonstrate  the  application  to  students  or  students  do  not 
properly  understand  the  use  and  value  of  the  application. 

(3)  Information  Representation:  It  is  critical  to  present  material  in  a  manner  to  which 
students  are  receptive  (e.g.,  visually  rich  media,  video,  audio). 

(4)  User  Interaction:  Users  interact  with  applications  in  a  variety  of  ways,  such  as 
voice  interaction,  touch  sensitive  interaction,  text  entry,  and  visual  interaction. 
Selecting  the  interaction  type  requires  considerations  of  time,  budget,  access  to 
advanced  technologies  and  the  ability  to  ensure  real-world  applicability  for  users. 

(5)  Usability:  Applications  must  be  user-friendly,  functional,  and  engaging  in  order  to 
effectively  reach  users. 

(6)  Utility  -  Value:  In  order  to  be  of  value,  applications  must  deliver  cost-effective  and 
meaningful  training  outcomes,  (e.g.,  performance  improvement,  skill  acquisition). 

(7)  Compatibility:  Applications  should  be  developed  to  run  equally  well  on  multiple 
operating  systems. 

Solutions:  To  increase  chances  of  successful  application  development  and  use,  consider  the 

following  actions  and  practices  associated  with  the  above  issues: 

(1)  Development:  Ensure  that  identified  resources  and  your  initial  development  plan 
sufficiently  clarify  and  address  your  training  requirements,  content  focus,  and 
established  Army  standards  for  application  development. 

(2)  Implementation:  Ensure  instructors  take  care  in  fully  introducing  applications  to 
students. 

(3)  Information  Representation:  Utilize  rich  media  content,  video  and  audio.  Limit  text- 
rich  documents  and  PowerPoint  presentations  when  possible;  consider  using  a 
game-style  approach. 

(4)  User  Interaction:  Incorporate  interactive  features  like  accelerometers  and  leverage 
other  handheld  device  functions  to  improve  user  interaction. 

(5)  Usability:  Ensure  that  the  application  functions  properly  and  is  user-friendly;  make 
the  application  engaging. 

(6)  Utility  -  Value:  Identify  quantitative  measures  to  track  use,  student 
performance/accomplishment  of  designed  activities,  frequency  of  function 
problems/outages,  and  repair  and  replacement  rates  to  determine  whether  the 
application  is  cost-effective  and  contributing  to  desired  training  outcomes. 

(7)  Compatibility:  Consider  required  operating  systems  and  use  cross-platform  tools  to 
develop  applications  for  use  across  multiple  operating  systems;  obtain  appropriate 
licenses  and  factor  in  related  costs  and  approval  times,  especially  regarding  any 
network  requirements  or  need  to  establish  stand-alone  systems. 
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Metafactors:  Training  Methods  and  Delivery 
Factor:  Training  Context 


Description:  This  factor  focuses  on  issues  central  to  the  training  environment. 


Issues:  Context  can  have  a  significant  impact  on  training.  Consider  the  following  issues  with 

various  training  contexts: 

(1)  Classroom:  Handheld  mobile  devices  can  be  a  source  of  distraction  in  the 
classroom;  learners  can  become  dependent  on  device  memory  instead  of 
developing  their  own. 

(2)  Field/Ranqe:  Environmental  conditions  may  not  be  conducive  to  proper  device 
use;  handheld  mobile  devices  could  detract  from  performance  during  training, 
particularly  under  high-risk  and  dangerous  situations. 

(3)  During  Personal  Time:  "Self-directed"  learning  requires  a  more  careful  program  of 
assessment  and  evaluation  to  ensure  consistency  with  objectives  and  content 
provided  in  other  ways. 

Solutions:  To  increase  chances  for  success  in  the  above  training  contexts,  apply  the  following 

practices  associated  with  the  different  contexts: 

(1)  Instructors  should  help  determine  whether  and  when  handheld  mobile  device  use 
is  appropriate  for  the  training  (see  Instructors  factor). 

(2)  Ensure  that  the  devices  can  survive  the  actual  training  environment  and  that 
users  have  complete  knowledge  of  their  appropriate  use  and  control. 

(3)  Ensure  clear  and  detailed  instructions,  guidance,  and  usage  policy  are  provided 
and  model  appropriate  use  throughout  the  installation/base,  so  users  understand 
how  to  appropriately  use  the  devices  and  applications  on  their  own  time. 
Instructors  can  demonstrate  the  value  of  using  the  handheld  device  by  modeling 
how  applications  and  handheld  devices  should  be  used  during  personal  time  to 
review  or  learn  more. 
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Metafactor:  Training  Methods  and  Delivery 
Factor:  Training  Content 


Description:  This  factor  focuses  on  the  development  of  training  applications,  including  lessons  learned 
and  recommendations  on  how  to  develop  applications  for  handheld  mobile  devices  (e.g.,  expected  time 
investments,  expected  barriers,  application  development  process  recommendations,  etc.). 


Issues:  Consider  the  following  issues  related  to  training  content: 

(1)  Training  objectives 

(2)  Complexity  and  accessibility  of  the  training  content 

(3)  Content  constancv/stabilitv  (e.g.,  rapidly  changing  content,  outdated  content. 

etc.') 

(4)  Individual/Collective  nature  of  training  tasks 

(5)  Training  Resource  Reguirements 

(6)  Training  Feedback 

(7)  Level  of  fidelity  of  training/assessment  environment 

(8)  Aided  vs.  unaided  instruction 

(9)  Inherent  training  risk  and  conseouences  of  performance  errors 


Solutions:  For  the  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Establish  specific,  verifiable  objectives  from  the  operational  environment. 

(2)  When  training  is  complex,  utilize  training  aids  that  will  support  trainees'  learning. 
Story-based,  game-like  scenarios  can  be  used  to  engage  learners  and  allow  them 
to  practice  performing  to  achieve  complex  learning  objectives. 

(3)  Ensure  content  is  current  and  software  is  up-to-date. 

(4)  Handheld  mobile  devices  can  support  realistic  trainings  for  both  individual  and 
collective  tasks,  if  appropriate  measures  and  developments  are  implemented. 

(5)  Handheld  mobile  devices  can  provide  effective  virtual  training  experiences  when 
real-world  equipment  is  not  feasible. 

(6)  Handheld  mobile  device  feedback  must  be  consistent  with  feedback  from  other 
sources  to  protect  against  introducing  confusion;  student-to-student  feedback  can 
ease  instructor  burden. 

(7)  Tracking  learner  performance  via  the  device  can  ensure  learners  master  the 
fundamentals  before  utilizing  high-fidelity  simulations;  training  device's  fidelity  to 
the  psychological  and  physical  environment  should  adhere  to  Army  standards  and 
be  consistent  across  multiple  platforms. 

(8)  Digital  devices  can  support  Soldiers'  self-monitoring  of  performance  but  can  also 
be  used  as  a  training  aid  when  properly  aligned  with  on-the-job  tasks  provided 
that  they  are  used  appropriately  and  learners  do  not  become  dependent  on  the 
aid. 

(9)  Handheld  mobile  device  training  can  serve  as  an  effective  alternative  to  real-life 
training  in  situations  that  involve  dangerous  settings  or  high  risk. 
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Metafactor:  Human  and  Contextual  Factors 
.Factor:  Students _ 


Description:  This  factor  focuses  on  issues  related  to  students'  use  of  handheld  mobile  devices  and 
digital  applications. 


Issues:  Consider  the  following  issues  that  can  have  an  impact  on  student  use  of  handheld 

mobile  devices  or  digital  applications: 

(1)  Attitudes  and  experiences  with  devices  and  applications  and  readiness  for  self- 

directed  learning:  Individual  differences  among  students  lead  to  a  wide  variety  of 
experiences  and  preferences  related  to  mobile  training,  including  potential 
aversions  to  using  handheld  mobile  devices  and  applications,  lack  of  necessary 
confidence  or  maturity  for  use,  or  lack  of  familiarity  with  such  devices.  Students 
either  have  or  have  not  used  mobile  learning  devices  in  AIT;  those  who  have  not 
had  any  prior  experience  may  need  significant  training  on  the  device  and  training 
application. 

Solutions:  For  the  above  issues,  consult  the  solution  below: 

(1)  Provide  complete  training  on  both  the  device  and  the  application;  pair  up 
individuals  who  are  comfortable  with  the  technology  with  those  who  are  not; 
consider  different  learning  orientations;  establish  clear  guidelines,  policies,  and 
expectations  regarding  their  use  during  training. 
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Metafactor:  Human  and  Contextual  Factors 
Factor:  Instructors 


Description:  This  factor  focuses  on  issues  related  to  instructor  attitudes,  experiences,  and  preferences 
regarding  the  use  of  handheld  mobile  devices  and  digital  applications  in  training. 


Issues:  Consider  the  following  issues  related  to  instructors  in  the  handheld  mobile  device 

training  process: 

(1)  Experience  and  Attitudes  toward  handheld  mobile  technologies:  Instructors  might 
lack  experience  and/or  confidence  with  handheld  mobile  devices  or  have  an 
aversion  to  using  handheld  mobile  technologies  in  training. 

(2)  Device's  Role  in  Training:  Instructors  must  understand  the  role  of  the  device  in 
the  training  and  use  the  device  to  an  appropriate  extent. 

Solutions:  For  the  above  issues,  consult  the  solutions  below  based  upon  the  item  numbering: 

(1)  Include  instructors  in  development  of  applications,  train  instructors  on  effective 
use  of  the  device,  and  demonstrate  the  cost-effectiveness,  efficacy,  and  benefits 
of  employing  these  tools  in  training.  Actual  and  immediate  acceptance  by 
individual  instructors  of  these  tools  should  not  be  assumed. 

(2)  When  possible,  instructors  should  be  able  to  use  the  handheld  devices  to 

•  Access  course  resources 

•  Facilitate  social  networking 

•  Demonstrate  collaborative  applications 

•  Create  opportunities  for  enhancing  feedback  on  the  course,  training 
application,  and  instructor  effectiveness 

•  Track  course  metrics,  such  as  time  on  task  or  frequency  of  use. 
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Metafactor:  Human  and  Contextual  Factors 
Factor:  Institutional  Support 


Description:  This  factor  focuses  on  institutional  support  in  utilizing  handheld  mobile  devices  and 
applications  for  training. 


Issues:  Consider  the  following  issues  related  to  institutional  support  for  handheld  mobile 

device  training: 

(1)  Broad-based  support  in  the  Army's  training  culture:  The  Army  Learning  Model  and 
Army  Learning  Concept  2015  emphasize  a  shift  toward  Army-wide  acceptance  of 
a  blended  learning  approach  incorporating  virtual  and  constructive  simulations, 
gaming  technology,  and  other  technology-delivered  instruction  in  training. 

(2)  Support  from  the  Chain  of  Command  for  using  handheld  mobile  devices  in  AIT  will 
influence  the  success  of  integration  efforts. 

Solutions:  For  the  above  issues,  consult  the  following  solutions: 

(1)  Identify  the  policies  that  will  require  planning,  work,  or  coordination  in  order  to 
achieve  compliance.  Describe  how  you  will  do  so.  Use  the  hyperlinked  references 
above  to  assist  in  the  development  of  the  case  for  use  of  handheld  mobile  devices 
and  digital  applications. 

(2)  Build  a  case  focusing  on  the  benefits  you  have  identified,  the  specific  training 
goals  and  scope  of  your  planned  employment,  and  a  complete  assessment  of  the 
projected  costs.  If  you  encounter  resistance,  identify  potential  partners  within  the 
Chain  of  Command  who  will  help  you  usher  in  mobile  learning  and  work  to 
advocate  for  change. 
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Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints 
Factor:  Device 


Description:  This  factor  focuses  on  device-related  issues  to  better  ensure  successful  integration  of 
devices  and  applications  into  AIT. 


Issues:  Consider  the  following  issues  related  to  devices  and  applications: 

(1)  Utilizing  device  capabilities  including  network  connectivity,  display,  media 
presentation,  peripherals  may  be  restrictive. 

(2)  Usability  of  the  device  including  user  friendliness,  human  factors,  and  associated 
training  required  for  effective  use  of  devices  should  be  considered  when  designing 
an  application. 

(3)  Durability  of  the  device  including  the  toughness  and  ability  to  withstand  various 
physical  conditions  (e.g.,  being  carried  in  a  cargo  pocket  while  Solider  runs,  being 
dropped  from  10  feet)  can  become  problematic  if  not  sufficiently  rugged. 

(4)  Portability  of  the  device  is  important  to  consider.  If  Soldier  transport  of  the 
device  (closely  tied  to  device  dimensions  and  weight)  is  difficult,  that  can  impede 
full  use. 

(5)  Supportabilitv  of  the  device  (i.e.,  the  ability  and  availability  of  resources  required 
to  support  various  handheld  mobile  devices  over  time,  including  the  ability  to 
maintain  and  upgrade  devices)  can  render  the  device  useless  if  not  fully  provided. 

(6)  Need  for  peripherals,  including  additional  peripherals  to  be  used  in  conjunction 
with  the  devices  to  deliver  training.  Peripherals  include  device  accessories  such  as 
battery  packs  for  extended  life,  SIM  cards,  memory  cards,  data  cables, 
headphones,  cameras,  etc. 


Solutions:  For  each  of  the  above  issues,  consult  the  following  solutions  based  on  the  item 

numbering: 

(1)  The  device  should  be  network  capable  and  fonts  and  graphics  should  be  easy  to 
read;  eliminate  unnecessary  code,  content,  and  operations. 

(2)  The  device  should  be  easy  to  use  and  users  should  be  trained  on  how  to  properly 
use  the  device. 

(3)  If  the  device  is  to  be  used  in  a  field  environment,  consider  additional  protective 
measures  such  as  a  carrying  case  or  a  screen  protector. 

(4)  Smaller  devices  tend  to  be  easier  to  carry,  but  they  must  also  fit  display  size 
requirements. 

(5)  Provide  access  to  networks  when  the  Army  network  is  restricted,  and  identify 
specific  service  and  repair  requirements,  especially  projected  upgrade  schedules, 
technology  life-spans,  and  replacement  options. 

(6)  CAC-enabled  devices  eliminate  the  need  for  encryption.  If  power  outlet  access  is 
limited,  carefully  consider  the  desired  battery  life  of  the  device. 


Page  13  of  59 


A  Practical  Decision  Guide  for  Integrating 
Mobile  Technologies  and  Digital  Applications  in  AIT 


Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints 
Factor:  Networking,  Security,  and  Implementation  Issues 


Description:  This  factor  focuses  on  networking,  security  and  implementation  issues  that  are  important 
to  consider  when  devices  will  require  network  access  and  data  use  and  transmission  are  involved. 


Issues:  Consider  the  following  issues  related  to  devices  and  applications: 

(1)  Networking  capabilities,  restrictions,  and  policy  requirements  associated  with 
connecting  and  using  handheld  mobile  devices  on  a  network. 

(2)  Security  level  of  training  information  for  a  specific  program  of  instruction  or  MOS 
to  be  used  on  handheld  mobile  devices. 

(3)  Device  and  application  security  to  ensure  the  security  of  both  the  device  and  the 
application,  the  ability  to  circumvent  these  security  measures,  and  the  longevity 
of  these  measures  (ex.  self-healing  devices  have  the  ability  to  remove  some 
security  safe-guards). 

Solutions:  For  the  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Pay  attention  to  the  restrictions  and  permissions  to  identify  how  to  access  an 
appropriate  network  for  the  training.  Not  all  applications  will  require  network 
access. 

(2)  If  the  training  content  contains  information  that  is  classified,  then  a  mobile 
platform  may  not  be  the  best  tool.  In  this  instance,  you  should  consider  the 
subfactors  (a)  networking  capabilities,  restrictions,  and  policy  requirements,  (b) 
device  and  application  security,  (c)  student  restrictions,  and  (d)  device 
management. 

(3)  Ensure  device  incorporates  all  possible  security  safeguards,  including  screen  lock, 
remote  wipe  programs,  encrypted  data,  and  secure  location  requirements. 
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Section  2:  Metafactors,  Factors  and  Subfactors  Worksheets 


How  to  use  Section  2  of  this  guide: 

For  each  of  the  following  Metafactors,  Factors  and  Subfactors,  the  user  should  print  out  each  Factor 
sheet  and  complete  the  checklists  and  other  information  related  to  identifying  the  most  significant 
issues  and  potential  challenges  related  to  each  subfactor  and  determine  the  relevancy  of  the  solution 
information  provided  as  it  relates  to  issues  and  challenges. 

Throughout  this  section,  issues  associated  with  the  subfactors  are  identified.  Where  specific  solutions 
or  best  practices  have  been  identified  for  specific  issues,  the  solution  numbers  correspond  to  the 
appropriate  issue  number.  Thus,  there  may  be  multiple  identically  numbered  solutions  offered  for  a 
single  issue.  Issue  numbers  follow  the  check  boxes  when  presented  in  checklist  format.  General  issues 
or  solutions  that  are  not  aligned  with  a  specific  one-to-one  issue-to-solution  system  are  frequently 
presented  in  bullet  format. 
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Metafactor:  Training  Methods  and  Delivery 


Factor:  Application 


Subfactor:  Development 


A  Practical  Decision  Guide  for  Integrating 
Mobile  Technologies  and  Digital  Applications  in  AIT 


(4)  Utilize  templates  to  ensure  consistency  with  Army  standards  for  graphical  user 
interfaces  (GUIs)  and  other  application  characteristics. 

More  successful  development  of  mobile  applications  for  AIT  can  be  also  supported  by 

the  following  practices: 

(5)  Utilize  cross-platform  development  tools  (such  as  those  provided  by  Sencha®) 
and  common  developer  programming  languages,  such  as  HTML5  and  CSS  to 
develop  a  mobile  application  that  will  function  equally  well  across  multiple 
platforms  (taking  into  account  the  need  to  acquire  licenses  for  such  tools  and  the 
associated  costs  and  approval  times). 

(6)  Avoid  using  software  development  kits  that  have  limited  cross-platform 
functionality.  If  possible  avoid  using  contractor  proprietary  code  because  when 
application  updates  are  required  an  additional  cost  may  be  incurred  or  there  may 
be  limitations  to  how  the  code  can  be  modified. 

(7)  Develop  user  interfaces  for  mobile  applications  by  incorporating  simplicity, 
optimal  screen  sizes,  and  controls  that  favor  two-handed  grips  and  thumb 
manipulation. 

(8)  Include  a  user  feature  that  gathers  student  and  instructor  comments,  which  can 

serve  as  the  basis  for  future  upgrades  to  the  application. _ 
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Description:  This  subfactor  focuses  on  the  implementation  of  applications  in  training,  including  training 
on  how  to  use  the  application  and  the  introduction  of  applications  to  students. 


Issues:  The  following  challenges  have  been  identified  as  significant  barriers  to  the  effective 

implementation  of  applications  during  AIT : 

□  (1)  Instructors  do  not  always  demonstrate  the  applications. 

□  (2)  Instructors  do  not  always  possess  the  expertise  needed  to  provide  effective 

device  and  application  training  to  students. 

□  (3)  Students  have  been  given  handheld  mobile  devices  and  told  to  learn  how  to 

use  them  by  "playing"  with  them  without  receiving  instruction. 

□  (4)  Instructors  have  distributed  instructional  booklets  to  teach  students  how  to 

use  the  applications  without  instructor-led  training. 


Solutions:  Successful  implementation  of  applications  in  AIT  can  be  enhanced  by  taking  the 

following  actions: 

(1)  Ensure  that  instructors  fully  demonstrate  devices  and  applications  for  students. 

(2)  Ensure  that  all  instructors  have  received  training  in  the  use  of  devices  and 
applications.  For  example,  instructors  need  to  receive  training  on  how  to  operate 
applications  and  functionality  and  go  beyond  simple  user  orientation.  This 
advanced  information  should  provide  instructors  with  the  expertise  they'll  require 
to  deliver  instruction  via  applications  and  mobile  devices  to  students.  Instructor 
training  should  link  course  objectives  to  application  content  and  suggest  methods 
for  instructors  to  leverage  applications  to  meet  course  objectives. 

(3)  Do  not  assume  that  students  can  or  will  learn  how  to  use  devices  and  applications 
independently. 

(4)  Prevent  instructors  from  using  instructional  booklets  without  support  from 
instructor-led  training. 
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Metafactor:  Training  Methods  and  Delivery 
Factor:  Application 

Subfactor:  Information  Representation 


Description:  This  subfactor  focuses  on  how  information  should  be  presented  in  applications.  Topics 
within  this  subfactor  may  include  considerations  for  text-based  vs.  graphic-based  vs.  audio-based 
information  presentation,  the  use  of  scaffolding,  and  linear  vs.  adaptive  progression. 


Issues:  In  order  to  be  effective,  training  on  mobile  devices  must  be  engaging  for  students,  so 

the  following  should  be  considered: 

□  (1)  Students  have  expressed  a  preference  for  rich  media  content  over  text 

documents  and  PowerPoint  presentations. 

□  (2)  Keeping  text  documents  and  PowerPoint  presentations  updated  can  be 

challenging. 


Solutions:  Implementing  the  following  practices  when  developing  training  applications  could 

improve  student  response  and  training  effectiveness  outcomes: 

(1)  Use  more  video  and  audio. 

(1)  Minimize  the  use  of  text  documents  and  PowerPoint  presentations. 

(1)  When  including  online  resources,  provide  alternatives  within  the  application,  such 
as  allowing  the  application  to  be  stored  entirely  self-contained  on  the  device,  in 
case  of  unreliable  network  access. 

(1)  Training-related  audio  or  video  content  can  be  presented  as  podcasts  or  vodcasts. 
which  are  downloadable  to  handheld  mobile  devices  as  well  as  laptop  and  desktop 
computers. 

(1)  Consider  presenting  information  in  the  form  of  a  game  with  rewards  (e.g.,  points, 
medals,  etc.)  for  engaging  learning. 

(2)  When  using  text  elements,  incorporate  tabs  and  hyperlinks  to  make  them  more 
interactive 

(2)  When  including  training  content  that  requires  frequent  updating,  consider 

developing  the  application  to  update  easily  by  accessing  external  online  resources 
(such  as  an  Army  schoolhouse's  website  listing  all  MOS  critical  tasks)  when 
prompted.  This  will  reduce  the  burden  on  the  application  developers  to  maintain 
and  update  the  application  for  rapidly  changing  content,  as  the  burden  for 
maintaining  up-to-date  content  falls  on  the  producers  of  the  Army  content. 
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Metafactor:  Training  Methods  and  Delivery 
Factor:  Application 
Subfactor:  User  Interaction 


Description:  This  subfactor  focuses  on  how  users  will  interact  with  applications.  Considerations  may 
include  the  necessity  of  voice  interaction,  haptic  interaction  via  a  touch  screen,  text  entry,  visual 
interaction  (e.g.,  viewing  training  multimedia),  collaboration  between  students,  or  network  interaction. 


Issues:  When  selecting  user  interaction  features  for  an  AIT  application,  the  following 

challenges  should  be  considered: 

□  (1)  The  development  of  complicated  applications  for  use  in  AIT  can  take  years  and 

cost  millions  of  dollars. 

□  (2)  Current  Army  training  developer  capabilities  limit  how  quickly  AIT  applications 

can  be  developed  and  how  sophisticated  they  are. 

□  (3)  Users  expect  application  quality  and  features,  such  as  incorporating  user 

movement,  to  match  those  of  advanced  commercial  gaming  and  simulations. 


Solutions:  The  following  factors  could  lead  to  improved  user  interaction  with  applications 

developed  for  AIT : 

(1)  Tradeoffs  between  device  costs,  capabilities  and  training  requirements  should  be 
well  thought  out  to  ensure  current  training  needs  are  being  addressed  through 
cost  effective  devices  and  applications. 

(2)  Work  with  developers  to  understand  their  capabilities  and  explore  alternative 
sources  (e.g.,  contracting)  to  develop  required  applications. 

(3)  In  some  instances,  incorporating  user  movement  by  integrating  accelerometers 
(which  register  the  motion  of  handheld  devices),  can  improve  training  fidelity  and 
help  develop  psychomotor  skills,  as  demonstrated  by  game  systems  like  the 
Wii®,  Xbox®,  and  PlayStation®. 

(3)  When  incorporating  user  movement,  tool/system  interface  fidelity  is  crucial  to 
preventing  negative  behaviors  (i.e.,  training  applications  need  to  replicate  real  life 
conditions  and  processes  so  learners  do  not  establish  incorrect  habits  or  learn 
something  that  would  cause  poor  performance  in  the  real  world). 
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Metafactor:  Training  Methods  and  Delivery 
Factor:  Application 
Subfactor:  Usability 


Description:  This  subfactor  focuses  on  the  user  friendliness  (an  aspect  of  usability  which  can  negatively 
impact  user  acceptance  of  an  application)  of  applications  and  other  aspects  such  as  application 
functionality  and  user  interface. 


Issues:  A  lack  of  user-friendliness  can  result  from  the  following  factors: 

□  (1)  An  inability  of  the  application  to  function  as  advertised. 

□  (2)  An  inaccessibility  of  controls  and  functions. 

□  (3)  A  slow  rate  of  device  or  application  operation. 

□  (4)  Frequent  malfunctions  and  crashes. 

Students'  motivation  to  use  a  training  application  can  be  negatively  affected  by  the 
following  factors: 

□  (5)  Users  are  bored  by  the  application. 

□  (6)  There  is  limited  or  no  positive  reinforcement  for  users. 


Solutions:  The  following  practices  could  enhance  the  user-friendliness,  and  therefore  the 

usability,  of  a  mobile  application  developed  for  AIT: 

(1)  Ensure  that  the  application  functions  properly,  as  demonstrated  by  thorough 
testing. 

(2)  Make  the  controls  and  navigation  simple  and  intuitive. 

(3)  Ensure  that  the  application  executes  instructions  in  a  timely  manner. 

(4)  Ensure  that  the  application  has  been  thoroughly  tested  and  is  free  of  frequent 
malfunctions  and  crashes. 

The  following  practices  could  improve  students'  motivation  to  use  applications 

developed  for  AIT : 

(5)  Make  the  application  fun  and  entertaining. 

(6)  Emphasize  similarities  to  parallel  games  and  activities  developed  for  commercial 
use. 

(7)  Provide  rewards  within  the  application  (e.g.,  scores,  medals,  bonus  points,  etc.) 
to  encourage  use  stimulated  by  competition  against  self  and  others. 
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Description:  This  subfactor  focuses  on  the  value  associated  with  applications,  such  as  improved 
motivation  of  trainees,  improved  skill  transfer  to  the  job,  reduction  in  errors,  etc. 


Issues:  The  following  points  are  relevant  to  the  demonstration  of  application  utility: 

□  (1)  Utility  is  seen  as  one  of  the  most  important  aspects  of  mobile  learning,  as  a 

critical  factor  if  the  degree  to  which  the  application/handheld  mobile  device 
actually  improves  training  effectiveness  and  remains  cost  effective. 

□  (2)  Instructors  must  be  convinced  of  the  training  effectiveness  of  mobile  learning 

applications  for  AIT. 

□  (3)  Instructors  must  be  convinced  of  the  cost  effectiveness  of  mobile  learning 

applications  for  AIT. 


Solutions:  When  determining  the  utility  of  mobile  learning  applications,  the  following  factors 

should  be  considered: 

(1,  2)  Quantifiable  data  is  important  in  considering  training  effectiveness  (e.g., 
whether  students  training  with  an  application  receive  higher  grades  than 
students  without). 

(1,  2)  Demonstrations  of  training  effectiveness  should  highlight  specific  capabilities 
designed  to  enhance  student  performance. 

(1,  3)  The  use  of  an  application  can  lead  to  cost  savings  (e.g.,  reduced  paper  cost 
due  to  less  need  for  paper-printed  training  manuals). 

(1,  3)  The  use  of  an  application  can  reduce  needed  training  time  on  equipment  with 
limited  availability;  students  can  practice  skills  on  an  application  rather  than 
the  actual  equipment,  thus  potentially  reducing  overall  training  time  and 
associated  costs. 

(1,  3)  The  use  of  an  application  can  reduce  the  risk  of  injury  and  other  negative 
outcomes  while  students  master  potentially  dangerous  skills. 

(1,  3)  Maximizing  the  use  of  an  application  throughout  multiple  training  segments 
could  increase  the  return  on  investment. 
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Metafactor:  Training  Methods  and  Delivery 
Factor:  Application 
Subfactor:  Compatibility 


Description:  This  subfactor  focuses  on  the  compatibility  of  applications  with  one  or  multiple  devices  or 
operating  systems. 


Issues:  A  challenge  with  developing  effective  applications  is  that  there  may  be  multiple 

devices  used  across  students: 

□  (1)  Depending  on  the  specific  devices  available  to  students  for  AIT,  applications 
should  be  developed  to  run  equally  well  on  multiple  operating  systems,  such 
as  Android,  Apple,  and  RIM  (Blackberry). 


Solutions:  The  following  factors  should  be  considered  to  better  ensure  the  timely  and  cost- 

effective  development  of  mobile  learning  applications  that  are  compatible  with 
multiple  devices  or  operating  systems: 

(1)  The  devices  and  associated  operating  systems  available  for  AIT  should  be 
determined. 

(1)  A  cross-platform  software  development  kit  fSDK~)  and  other  cross-platform  tools 
should  be  used  to  develop  one  application  for  use  across  multiple  operating 
systems. 

(1)  Appropriate  enterprise/developer  and  distribution  licenses  must  be  obtained  for 
the  cross-platform  SDK  and  other  tools. 

(1)  Costs  and  approval  times  must  be  taken  into  account  when  acquiring  licenses 
(e.g.,  acquiring  an  enterprise  license  can  take  up  to  14  months). 
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Description:  This  subfactor  focuses  on  the  use  of  handheld  mobile  devices  and  training  applications  in 
the  classroom. 


Issues:  Indicate  by  checking  the  items  below  which  issues  may  be  relevant  when  considering 

using  handheld  mobile  devices  in  the  classroom. 

□  (1)  Handheld  mobile  devices  would  be  a  source  of  distraction  in  the  classroom. 

□  (2)  Learners  can  become  dependent  on  the  memory  capacity  of  handheld  mobile 

devices  and  fail  to  develop  their  own,  internal  capabilities. 

□  (3)  Learners  can  become  dependent  on  the  device's  representation  capabilities 

(e.g.,  graphing,  charting,  image  capture)  instead  of  building  their  own  mental 
models. 


Solutions:  The  following  solutions  should  be  used  to  address  the  issues  above  concerning  the 

use  of  handheld  mobile  devices  and  training  applications  in  a  classroom  training 
context 

(1,  2,  3)  Instructors  should  determine  when  the  use  of  handheld  mobile  devices  is 
appropriate.  For  example,  in  the  classroom  instructors  could  tell  students 
when  they  are  allowed  to  use  devices  and  when  they  are  not.  This  would 
help  to  keep  the  students  from  becoming  distracted  by  their  handheld 
device  when  their  attention  should  be  focused  elsewhere. 

(1,  2,  3)  Use  of  handheld  mobile  devices  should  be  delayed  until  learners  have 
demonstrated  mastery  of  basic  material. 

(3)  Instructors  may  require  learners  to  demonstrate  understanding  independently 
from  the  handheld  mobile  device  until  they  are  satisfied  that  mastery  has  been 
achieved. 
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Description:  This  subfactor  focuses  on  the  use  of  handheld  mobile  devices  and  training  applications  in 
a  field/range  context. 


Issues:  Using  handheld  mobile  devices  during  field  and  range  training  and  exercises  needs 

special  consideration  given  these  environments. 

Indicate  by  checking  the  items  below  which  issues  may  be  relevant  when  considering 
using  handheld  mobile  devices  during  field  and/or  range  training  and  exercises. 

□  (1)  The  environmental  and  use  conditions,  including  weather  and  student 

movements,  may  not  be  conducive  to  proper  functioning  and  device  durability, 
placing  undue  stress  on  these  devices. 

□  (2)  Use  of  the  devices  could  detract  from  performance  during  field  and  range 

training  particularly  under  dangerous  situations. 


Solutions:  The  following  solutions  should  be  used  to  address  the  issues  concerning  the  use  of 

handheld  mobile  devices  and  training  applications  in  a  field/range  context. 

(1)  The  purchase  of  inexpensive  handheld  mobile  devices  may  require  more  rugged 
equipment  or  additional  protective  measures,  e.g.  covers,  containers,  etc. 

(1,  2)  Provide  detailed  instructions  on  where  and  when  the  use  of  these  devices  is 
appropriate  under  field  and  range  conditions. 
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Description:  This  subfactor  focuses  on  the  use  of  handheld  mobile  devices  and  training  applications  in 
one's  free  time  as  self-development,  e.g.  during  free  time  in  the  barracks  or  dead  time  between 
training  sessions. 


Issue:  The  central  issue  concerning  self-directed  learning  on  one's  own  time  is  described 

below. 

□  (1)  "Self-directed"  learning  requires  a  more-careful  program  of  assessment 
and  evaluation  to  ensure  that  this  learning  is  consistent  with  objectives 
and  content  provided  in  other  ways  (classroom,  practice,  etc);  students 
should  not  draw  incorrect  conclusions  about  the  training  content. 


Solution:  The  solutions  that  are  presented  below  will  help  to  learners  to  use  applications  for 

self-directed  learning. 

(1)  Behavior  modeling  by  instructors  and  other  staff  in  and  out  of  the  classroom  is 
an  effective  mode  of  instruction  for  regarding  proper  device  use  and 
restrictions. 

(1)  Training  developers  need  to  use  subject  matter  experts  and  instructors  during 
development  and  implementation  to  ensure  that  applications  used  for  self- 
directed  learning  are  in  line  with  training  objectives  and  classroom  course 
content. 
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Description:  This  subfactor  focuses  on  the  objectives  of  training  based  on  the  POI  and  established 
training  goals.  The  focus  here  is  determining  what  the  training  is  intended  to  accomplish  (i.e., 
successful  trainees  are  those  who  are  able  to...?). 


Issues:  Consider  the  following  issues  related  to  training  objectives: 

□  (1)  Objectives  can  be  expressed  in  vague  or  imprecise  wording. 

□  (2)  Collective  tasks  can  be  difficult  to  assess,  given  that  different  individuals  offer 

different  levels  of  commitment  and  expertise  to  task  performance. 

□  (3)  There  can  be  a  tendency  for  SMEs  to  specify  content  as  objectives  that  had 

been  critical  in  the  past  but  over  time  these  objectives  may  become 
automated  or  revised  by  new  procedures. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Objectives  should  be  expressed  as  observable  behaviors,  with  a  standard  of 
mastery  and  a  time  limit.  This  will  support  the  development  of  mobile 
applications  that  are  responsive  to  operational  requirements. 

(2)  Collective  job-performance  tasks  should  be  assessed  both  at  the  sub-component 
individual  task  level  (for  individual  performance)  and  at  the  collective  task  level 
(for  team  performance).  Careful  development  of  the  application  will  allow  for 
learners  to  observe  alignment  of  individual  and  team  performance. 

(3)  Whenever  possible,  the  necessity  of  including  an  instructional  objective  should  be 
verified  by  direct  observation  of  job  performance  in  the  field  to  ensure  accuracy  of 
the  application.  Hand-held  devices  support  this  form  of  data  collection. 
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Description:  This  subfactor  focuses  on  the  complexity/simplicity  of  content  for  the  specific  task  being 
considered  for  the  employment  of  handheld  devices  and  digital  applications. 


Issues:  Consider  the  below  issues  related  to  complexity  of  training  content: 

□  (1)  Highly  complex  content  (e.g.,  detailed  or  counterintuitive  procedures, 

problem-solving)  requires  different  instructional  strategies  than  simple 
information  learning. 

□  (2)  Students  may  perceive  content  that  appears  to  be  simple  as  less  important 

than  more  difficult  material. 

□  (3)  Simple  content  may  also  be  less  engaging  content. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Highly  complex  tasks  sometimes  can  be  addressed  by  job  re-engineering  or  the 
creation  of  job  aids  that  can  be  incorporated  into  the  application. 

(2)  Operational  scenarios  and  story-based  instruction,  useful  in  conveying  the 
importance  of  content  that  appears  to  be  simple  or  trivial,  can  be  incorporated 
into  the  application. 

(3)  Competitive  game  play  can  be  used  to  increase  learners'  cognitive  engagement 
with  information  that  appears  to  be  relatively  simple  or  unimportant.  Learners 
can  compete  with  one  another  on  carefully  constructed  knowledge  checks  and/or 
training  metrics  using  the  device. 
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Description:  This  subfactor  focuses  on  the  constancy  and  stability  of  the  training  content  as  it  relates 
to  a  particular  MOS  or  program  of  instruction  (POI).  For  instance,  rapidly  changing  or  dynamic 
operational  environments  may  require  nearly  continuous  training  content  updates  and  easy  editing 
capabilities. 


Issues:  Consider  the  below  issues  related  to  stability  of  training  content: 

□  (1)  Subject-matter  content  can  become  obsolete  if  not  updated  regularly. 

□  (2)  Rapidly  changing  content,  subject  to  frequent  updates,  risks  factual  errors. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Personnel  should  record  and  consider  the  digital  applications' original  creation 
dates  to  help  determine  if  its  content  is  still  current. 

(2)  To  ensure  stability  of  updates  to  the  application,  need  to  develop  standardized 
development  and  monitoring  processes  that  include  routine,  independent 
validation  and  verification  of  content  accuracy  and  currency. 

(2)  Identify  the  extent  of  available  content  editing  capabilities  and  ensure 

appropriate  personnel  have  the  training  and  software  needed  to  edit  or  update 
the  training  application  as  needed. 
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Description:  This  subfactor  focuses  on  whether  or  not  the  application  is  for  individual  or  collective 
tasks. 


Issues:  Consider  the  following  issues  related  to  individual/collective  tasks  within  the  training: 

□  (1)  Realistic  training  scenarios  are  required  for  training  behaviors  to  generalize  to 

on-the-job  performance. 

□  (2)  On-the-job  performance  requires  team  coordination. 

□  (3)  Multi-player  applications  that  support  team-building  risk  losing  clear  focus  on 

individuals'  jobs. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Handheld  mobile  devices  support  more  immersive  training/rehearsal  scenarios 
than  possible  in  classroom  situations. 

(2)  Handheld  mobile  devices  can  support  collective  (i.e.,  "team")  performance. 

(3)  Single-player  applications  may  better  allow  learners  to  focus  on  learning  their 
own,  particular  jobs. 
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Description:  This  subfactor  focuses  on  the  resource  requirements  (e.g.,  equipment  or  range) 
associated  with  training.  As  an  example,  training  that  necessitates  the  use  of  special  equipment  would 
specify  this  equipment  as  a  resource  requirement  for  effective  training. 


Issues:  Consider  the  below  issues  related  to  training  content  resources: 

□  (1)  Job  performance  tasks,  especially  those  requiring  the  use  of  expensive, 

complicated,  or  hard-to-find  equipment  are  hard  to  train  in  regular 
classrooms. 

□  (2)  Different  schools  may  have  different  sets/volumes  of  resources. 

□  (3)  It  is  not  always  possible  for  instructors  to  provide  each  student  with  a  full  set 

of  instructional  resources. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Computer-based  simulations  delivered  on  handheld  mobile  devices  can  be 
effective  in  delivering  instruction  when  hands-on  exercises  with  operational 
equipment  are  not  feasible  (e.g.,  when  there  is  a  shortage  of  equipment,  or  the 
cost  of  training  using  an  application  would  be  more  efficient  than  using  physical 
resources). 

(2)  Instructors  must  be  permitted  to  tailor  instructional  methods  to  local  conditions. 
This  may  be  more  easily  accomplished  using  digital  applications  than  other 
methods. 

(3)  Using  a  collaborative  learning  approach,  students  can  share  resources  across 
applications,  supporting  the  goal  of  team-building. 
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Description:  This  subfactor  focuses  on  the  need  to  provide  students  feedback  during  training. 
Specifically,  determining  if  feedback  is  necessary  for  effective  training  progression  and/or  training 
outcomes  is  crucial.  If  it  is  necessary,  the  next  concern  is  determining  how  the  feedback  should  be 
delivered  (i.e.,  visual  vs.  audio,  instructor  vs.  machine,  progressively  or  at  the  end  of  training). 


Issues:  Consider  the  following  issues  related  to  feedback: 

□  (1)  Training  feedback  from  the  device  may  not  align  with  instructor  feedback. 

□  (2)  In  large  classes,  it  may  not  be  possible  for  instructors  to  provide  tailored 

feedback  to  each  learner. 

□  (3)  Training  feedback  from  a  device  can  be  inconsistent  with  operational  conditions 

such  as  noise  or  weather. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Determine  the  feasibility  and  nature  of  a  device  to  ensure  consistency  with 
instructor  feedback. 

(2)  Student-to-student  feedback  across  devices  can  ease  the  burden  on  the 
instructor  providing  feedback. 

(3)  Ensure  that  training  feedback  from  the  application  is  consistent  with  feedback 
from  other  sources. 
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Description:  This  subfactor  focuses  on  the  level  of  fidelity  associated  with  the  training  environment 
compared  to  the  execution  of  learned  tasks  in  the  practical  environment.  Specifically,  how  similar  is 
the  training  environment  to  the  real  environment  the  task  is  expected  to  be  performed  in  (e.g., 
immersive  flight  simulator  vs.  flying  a  plane).  Fidelity  could  relate  to  mundane  concepts,  such  as  does 
the  training  environment  look  similar  to  the  real  world  environment,  or  more  psychological  concepts, 
such  as  does  the  trainee  experience  similar  reactions  in  the  training  environment  as  in  the  real 
environment,  especially  in  terms  of  risk  management  and  time  pressures.  High-fidelity  is  a  very 
detailed  representation,  whereas  low-fidelity  is  a  less  detailed  representation. 


Issues:  Consider  the  following  issues  related  to  fidelity  of  the  training/assessment 

environment: 

□  (1)  Introducing  high-fidelity  instruction  and  assessment  can  be  confusing  in  its 

detail,  especially  early  in  the  curriculum,  before  students  have  experienced 
mastery  of  fundamentals. 

□  (2)  Device  design  could  be  inconsistent  with  Army  standards. 

□  (3)  Implementations  across  multiple  platforms  (e.g.,  Apple,  Android,  Blackberry) 

may  vary  in  how  the  equipment  is  represented. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Tracking  learner  performance  on  the  device  can  ensure  that  high-fidelity 
simulation  instruction  and  assessment  can  be  delayed  until  fundamentals  have 
been  mastered. 

(2)  Check  to  ensure  device  fidelity  is  consistent  with  applicable  Army  standards. 

(3)  Use  development  templates  to  ensure  consistent  device  fidelity  across  multiple 
platforms. 
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Description:  This  subfactor  focuses  on  whether  training  is  aided  or  unaided.  Specifically,  is  training 
delivery  accompanied  by  a  supporting  training  device  or  application?  And  if  a  supporting  training 
device  is  used,  how  should  the  training  device  be  used  and  what  limitations  need  to  be  placed  on  the 
use  of  the  device  to  prevent  dependency  on  the  training  aid. 


Issues:  Consider  the  following  issues  related  to  the  aided  or  unaided  nature  of  the  training: 

□  (1)  In  training  aided  by  training  support  devices,  learners  can  become  overly 

dependent  on  the  device,  devoting  more  attention  to  using  the  training  aid  and 
less  attention  to  the  actual  goals  of  instruction. 

□  (2)  Training  aids  that  dramatically  differ  from  actual  job  aids  can  hinder 

transference  to  on-the-job  performance. 

□  (3)  Poorly  designed  user  interfaces  can  impair  performance. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Incorporate  knowledge  checks  and  other  performance  metrics  that  encourage 
self-monitoring  and  reinforce  established  instructional  goals. 

(2)  Software  development  templates  and  appropriate  usage  limitations  can  ensure 
that  when  the  device  is  used  as  a  training  aid,  it  is  properly  aligned  with  actual 
job  aid  functionality  and  availability. 

(3)  Incorporate  simplicity  in  the  training  application's  design  and  the  capabilities 
desired  in  a  handheld  device,  with  smaller  screens  and  controls  that  favor  two- 
handed  grips  and  thumb  manipulation. 
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Description:  This  subfactor  focuses  on  whether  there  is  any  physical  danger  or  risk  associated  with 
training  and  the  consequences  of  committing  an  error  in  training.  Developing  applications  tasks  with 
serious/dangerous  consequences  if  errors  are  made  may  require  a  higher  level  of  fidelity  than  non- 
dangerous  tasks.  At  a  minimum,  training  should  include  an  emphasis  that  increases  the  learner's 
perceptions  of  the  risks  involved  and  the  consequences  of  inappropriate  or  erroneous 
actions/decisions. 


Issues:  Consider  the  following  potential  risks  associated  with  training  delivery: 

□  (1)  In  some  job-performance  tasks,  there  is  no  escape  from  the  possibility  of 

physical  injury. 

□  (2)  In  operating  complex  equipment,  or  for  job-performance  tasks  involving 

danger  (e.g.,  the  use  of  ordnance),  consequences  of  error  can  be  catastrophic. 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Device-delivered  job  rehearsal  can  be  effective  in  imparting  knowledge,  skills  and 
abilities  required  for  job  tasks  that  have  a  catastrophic  consequence  of  error. 
Using  handheld  mobile  devices  learners  can  learn  concepts  and  practice  skills  in  a 
safe  and  simulated  environment  before  practicing  in  more  dangerous 
environments. 

(2)  Device-delivered  mission  rehearsal  in  high-fidelity  simulations  can  be  useful  when 
learning  training  content  with  serious  risk  of  physical  injury. 
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Metafactor:  Human  and  Contextual  Factors 


Factor:  Students 


Subfactor:  Student  general  characteristics,  individual  differences,  learning  orientation 
attitudes  toward  device,  experiences,  and  readiness  for  self-directed  learning 


Description:  This  subfactor  focuses  on  the  general  characteristics,  individual  differences,  and  learning 
orientation  of  students  in  AIT.  Examples  of  characteristics,  individual  differences,  and  learning 
orientations  include,  but  are  not  limited  to:  age,  self-efficacy,  and  goal-setting  skills,  respectively.  In 
addition,  this  subfactor  concerns  AIT  students'  attitudes  (i.e.,  preferences  for  or  against  a  particular 
handheld  mobile  device  or  handheld  mobile  devices  in  general),  previous  experiences  with  handheld 
mobile  devices  (i.e.,  number  of  handheld  mobile  device  experiences  and  valence  of  those 
experiences),  and  readiness  for  self-directed  learning  (i.e.,  are  the  students  motivated  to  use  devices 
and  applications  on  their  own  time  to  enhance  knowledge  and  skills). 


Issues:  Which  of  the  following  may  be  issues  related  to  students'  use  of  handheld  mobile 

devices  or  digital  applications?  (Check  all  that  apply.) 

□  (1)  Aversion  to  use  of  device  or  applications. 

□  (2)  Lack  of  motivation  to  learn  how  to  properly  use  the  device  or  application. 

□  (3)  Students  not  mature  enough  to  use  the  device  for  learning  purposes. 

□  (4)  Students  lack  confidence  in  using  device  or  applications. 

□  (5)  Negative  attitude  toward  being  responsible  for  the  device  (i.e.,  high  value 

item;  worried  about  breaking  or  losing  it). 

□  (6)  Students'  desire  for  privacy  and/or  anonymity  when  using  the  device. 

□  (7)  Lack  of  familiarity  with  handheld  mobile  devices  and  capabilities. 


Solutions:  For  the  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1,  3,  4,  6,  7)  Have  those  individuals  who  are  good  at  using  the  device  model  it  for 
others;  pair  up  individuals  who  are  comfortable  with  the  technology  with  those 
who  are  not. 

(2,  6)  Help  the  Soldier  to  see  the  value  of  using  the  device  or  application  during 
training. 

(3,  7)  Provide  training  on  how  and  when  to  (and  when  not  to)  use  the  device  during 
training  and  follow-through  with  any  identified  restrictions. 

(5)  Increase  durability  of  device  (e.g.,  screen  protectors,  cases). 

(7)  Establish  clear  guidelines  and  policies  regarding  privacy  and  security  while  using 
the  device;  Educate  students  on  how  to  use  the  device  appropriately;  Ensure 
proper  privacy  and  security  controls  are  in  place  on  the  device  (e.g.,  firewalls, 
passwords,  etc.). 
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Factor:  Instructors 


Subfactor:  Experience  and  Attitudes  toward  Handheld  mobile  technologies 


A  Practical  Decision  Guide  for  Integrating 
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Metafactor:  Human  and  Contextual  Factors 


Factor:  Instructors 

Subfactor:  Device  Role  in  Training 


Description:  This  subfactor  focuses  on  identifying  exactly  what  purpose  or  role  the  handheld  mobile 
device  and  application  will  serve  during  training.  Examples  of  device  roles  include  supplementing  a 
lecture,  as  a  communication  device  for  instructors  to  contact  student,  or  use  as  a  demonstration  or 
practice  of  necessary  skills  (i.e.,  a  simulation). 


Issues:  When  deciding  how  best  to  use  mobile  applications  during  AIT,  keep  the  following 

points  in  mind: 

□  (1)  Students  expect  the  use  of  online  resources,  social  networking,  and 

collaborative  tools  (e.g.,  Blackboard  Collaborate)  during  training. 

□  (2)  A  lack  of  user  feedback  can  limit  the  development  of  effective  upgrades  to  an 

application. 

□  (3)  When  students  use  applications  during  training,  their  performance  metrics  are 

not  always  retrieved  and  used  to  evaluate  progress  and  training  effectiveness. 

When  planning  to  use  mobile  applications  instead  of  hands-on  training  with 
equipment,  consider  the  following  points: 

□  (4)  The  availability  of  equipment  for  hands-on  training  can  be  limited. 

□  (5)  Practicing  with  mobile  applications  that  simulate  hands-on  use  of  equipment 

can  improve  student  performance  with  that  equipment. 

□  (6)  Relying  too  heavily  on  mobile  applications  for  training  can  negatively  impact 

long-term  performance,  as  applications  cannot  duplicate  the  experience  of 
hands-on  training  in  every  possible  way. 


Solutions:  The  following  practices  could  help  in  selecting  appropriate  roles  for  mobile 

applications  during  AIT: 

(1)  When  possible,  and  enabled  by  the  application,  incorporate  online  resources, 
social  networking,  and  collaborative  tools  during  training. 

(2)  Track  student  feedback  provided  via  social  networking  and  collaborative  tools, 
then  apply  it  to  enhancing  training  and  upgrading  the  application. 

(3)  When  possible,  use  the  application  to  track  student  performance  (e.g.,  frequency 
of  use,  duration  of  self-study,  areas  of  difficulty,  etc.),  then  apply  them  to 
evaluating  student  progress,  enhancing  training,  and  upgrading  the  application. 

To  make  the  most  of  application-based  training  as  it  relates  to  hands-on  training  with 

equipment,  consider  implementing  the  following  practices: 

(4)  Use  applications  to  augment  hands-on  training,  not  replace  it. 

(5)  Encourage  students  to  practice  with  applications  when  they  have  downtime  or 
when  hands-on  training  is  unavailable. 

(6)  Prepare  students  for  the  differences  they  will  encounter  between  application- 
based  and  hands-on  training  with  actual  equipment  (e.g.,  the  occurrence  of 

_ problems  or  malfunctions). _ 
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Metafactor:  Human  and  Contextual  Factors 
Factor:  Instructors 

Subfactor:  Perceived  benefits  to  instructor 


Description:  This  subfactor  focuses  on  the  potential  benefits  that  instructors  can  expect  with  the  use  of 
handheld  mobile  devices  in  AIT. 


Issues:  Consider  the  following  related  to  instructors'  experience: 

□  (1)  Instructors  who  lack  familiarity  with  handheld  mobile  devices  may 

demonstrate  a  lack  of  buy-in.  Identify  how  many  of  your  instructors  will  need 
to  have  the  value  of  mobile  learning  demonstrated  to  them  in  order  to  buy  in 
to  the  process  to  determine  the  number  solutions  you  may  need  to  consider. 


Solutions:  In  order  to  get  buy-in  from  the  instructors  you  can  do  some  or  all  of  the  following 

exercises.  Indicate  which  exercises  you  think  will  be  required  to  get  buy-in  from 

instructors: 

(1)  Include  instructors  in  the  development  of  applications  and  selection  of  content. 

(1)  Demonstrate  benefits  of  increased  functionality  for  instructors. 

(1)  Demonstrate  ability  to  use  networking  to  connect  instructors  and  students  for 
collaborative  learning  and  self-directed  learning. 

(1)  Demonstrate  to  instructors  how  the  devices  increase  efficiency  and  make 
instruction  easier  for  them. 

(1)  Demonstrate  the  ability  of  mobile  learning  to  improve  student  learning. 

(1)  Demonstrate  the  cost-effectiveness  of  devices  to  instructors  in  terms  that  are 
meaningful  to  them,  such  as  impact  on  training  hours,  workload,  training 
flexibility,  etc. 
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Metafactor:  Human  and  Contextual  Factors 
Factor:  Institutional  Support 

Subfactor:  Broad-based  support  in  the  Army  training  culture 


Description:  This  subfactor  focuses  on  the  broad-based  support  in  the  Army  training  culture  for 
embracing  and  using  handheld  mobile  devices  in  AIT  or  training  in  general. 


Issues:  With  the  Army  Learning  Model  and  Army  Learning  Concept  for  2015  as  guidance, 

many  units  are  moving  toward  increased  use  of  handheld  mobile  devices  and  digital 
applications  in  training.  Multiple  organizations  offer  guidance  and  support  for  such 
efforts.  The  following  links  are  to  various  organizations  and  documents  that  contain 
policy  and  other  guidance  that  support  developing  and  implementing  mobile  learning 
effectively.  Identify  the  specific  policies  and  directives  that  support  your  vision  and 
goals. 


Solutions:  This  information  can  be  consulted  to  assist  in  building  a  case  for  using  devices  and 

application  and  guiding  implementation  through  best  practices.  (Note:  some  links 
may  not  remain  permanent  and  may  require  additional  searching  to  locate 
documentation  or  related  information.) 

•  TRADOC  Pam  525-8-3,  The  U.S.  Army  Training  Concept  2012-2020: 
http://www.tradoc.armv.mil/tpubs/pams/tp525-8-3.pdf 

•  SCORM  http://www.adlnet.gOv/capabilities/scorm/scorm-2004-4th#tab-main 

•  DoD  Instruction  (DoDI)  1322.26:  Development,  Management,  and  Delivery  of 
Distributed  Learning: 

http://www.dtic.mil/whs/directives/corres/pdf/132226p.pdf 

•  Advanced  Distributed  Learning  (ADL)  Initiative:  http://www.adlnet.org/ 

•  ADL  Mobile  Learning  Handbook: 
https://sites.google.eom/a/adlnet.gov/mobile-learning-guide/home/ 

•  Mobile  Learning  Approaches  for  U.S.  Army  Training,  Research  Note  2010-07, 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences: 
http://www.dtic.  mi  l/cgi-bin/GetTRDoc?AD=ADA528742 
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Metafactor:  Human  and  Contextual  Factors 
Factor:  Institutional  Support 

Subfactor:  Support  from  Chain  of  Command 


Description:  This  subfactor  focuses  on  support  from  the  Chain  of  Command  for  using  handheld  mobile 
devices  in  AIT. 


Issues:  Change  is  often  met  with  resistance.  There  may  be  individuals  from  various 

constituencies  who  will  resist  the  move  to  mobile  learning.  Which  of  the  following 
may  demonstrate  resistance?  (Check  all  that  apply) 

□  (1)  Instructors 

□  (2)  Training  Developers 

□  (3)  School  Leadership 


Solutions:  Identify  what  you  believe  to  be  the  motivation  behind  each  constituency's  resistance. 

Review  the  subfactor  information  contained  in  previous  guide  sections  related  to  each 
of  the  following: 

(1)  Instructors:  Metafactor:  Human  and  Contextual  Factors;  Factor: 
Instructors;  Subfactor:  Experience  and  Attitudes  toward  Handheld  mobile 
technologies 

(2)  Training  Developers  -  Metafactor:  Human  and  Contextual  Factors;  Factor: 
Institutional  Support;  Subfactor:  Broad-based  support  in  the  Army 
culture 

(3)  School  Leadership  -  Metafactor:  Human  and  Contextual  Factors;  Factor: 
Institutional  Support;  Subfactor:  Broad-based  support  in  the  Army 
culture 

(4)  Other  -  Determine  which  of  the  areas  covered  in  this  guide  may  assist  in  reducing 
resistance. 

You  will  likely  have  allies  within  the  Chain  of  Command  who  will  help  you  usher  in 
mobile  learning.  Here  are  some  additional  actions  you  may  take  to  improve 
implementation  success: 

•  Identify  the  leaders  and  contact  information  of  individuals  within  the  Chain  of 
Command  who  will  be  or  could  be  effective  advocates  for  change. 

•  Identify  what  you  will  do  to  unite  these  forces,  how  they  can  help  you  to  affect 
change,  and  what  evidence  is  needed  to  gain  their  support. 
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Description:  This  subfactor  focuses  on  how  to  get  the  most  out  of  device  capabilities  in  application 
development.  This  subfactor  identifies  issues  associated  with  various  handheld  mobile  devices  and 
application  development  practices  that  will  maximize  the  value  of  device  capabilities  for  mobile 
learning  in  AIT. 


Issues:  Check  the  box  next  to  each  of  the  following  potential  device  capability  issues  that  are 

identified  below,  where  the  issue  (or  lack  thereof)  may  present  a  challenge  to 
implementing  handheld  mobile  technologies  and  digital  applications  into  the  AIT 
course  under  consideration. 

□  (1)  Network  capabilities  for  user  collaboration 

□  (2)  Size  of  visual  display 

□  (3)  User  interface 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Select  a  device  that  is  network  capable  (have  internet  access  through  a  cell  signal 
or  through  wi-fi  capability).  Attempt  to  ensure  that  the  device  should  be 
supported  by  the  Army's  network  infrastructure. 

(3)  Fonts  and  graphics  should  be  easy  to  view. 

(3)  Pages  should  be  carefully  created  to  prevent  scrolling  in  two  dimensions. 

(3)  Unnecessary  code,  content,  and  operations  should  be  eliminated. 

(3)  Efficient  formats  and  standard  color  palettes  for  media  should  be  used. 

(3)  Content  should  be  divided  into  small  objects  for  easy  download. 

(2)  Larger  screens  may  be  preferred  as  they  tend  to  be  easier  for  Soldiers  to  view 
and  use.  Additionally,  larger  screens  may  be  required  to  manage  the  level  of 
information  presented  in  complex  applications  and  improve  user  interaction  with 
applications. 

(1-3)  Identify  the  issue  and  consult  information  throughout  this  guide  that  may  assist 
in  overcoming  any  potential  student  challenge  encountered. 
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Description:  This  subfactor  focuses  on  the  usability  of  handheld  mobile  devices  and  the  associated 
training  required  for  effective  use  of  devices. 


Issues:  Check  the  box  next  to  each  of  the  following  potential  device  usability  issues  that  are 

identified  below,  where  the  issue  (or  lack  thereof)  may  present  a  challenge  to 
implementing  handheld  mobile  technologies  and  digital  applications  into  the  AIT 
course  under  consideration. 

□  (1)  Ease  of  use 

□  (2)  Setting  for  device  use 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  Instructor  and  student  buy-in  will  be  easier  to  obtain  if  the  device  is  easy  to  use. 

(2)  Educate  the  Soldiers  on  how  to  use  the  device  properly; 

(2)  Determine  what  usage  guidelines  should  be  implemented  to  minimize  Soldier 
distraction  while  using  the  device; 

(2)  Determine  what  security  safeguards  should  be  activated  (or  turned  off)  on  the 
device  to  address  user  restrictions. 

(1-2)  Identify  the  issue  and  consult  information  throughout  this  guide  that  may  assist 
in  overcoming  any  potential  student  challenge  encountered. 
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Description:  This  subfactor  focuses  on  the  toughness  and  ability  to  withstand  various  physical 
conditions  of  various  handheld  mobile  devices  in  different  AIT  contexts. 


Issues:  Does  your  current  situation  require  a  device  that  can  be  used  in  a  field  environment? 

□  (1)  Yes 

□  (2)  No 


Solutions:  Consult  the  proposed  solutions  based  upon  the  item  numbering: 

(1)  Consider  additional  protective  measures  for  the  device  such  as  a  carrying  case  or 
screen  protector. 

(1)  If  additional  device  protection  or  the  cost  of  potential  damage  to  devices  is  cost 
prohibitive,  consider  limiting  the  use  of  devices  to  non-field  conditions  (e.g., 
classrooms,  designated  study  areas,  barracks). 

(2)  Durability  issues  may  not  be  of  principal  concern.  Standard  devices  should  suffice. 


Page  44  of  59 


A  Practical  Decision  Guide  for  Integrating 
Mobile  Technologies  and  Digital  Applications  in  AIT 


Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints 
Factor:  Device 

Subfactor:  Device  Portability 


Description:  This  subfactor  focuses  on  the  ease  of  transport  of  the  device. 


Issues:  Check  the  box  next  to  each  of  the  following  potential  device  portability  issues  that  are 

identified  below;  where  the  issue  (or  lack  thereof)  may  present  a  challenge  to 
implementing  handheld  mobile  technologies  and  digital  applications  into  the  AIT 
course  under  consideration. 

□  (1)  Lightweight 

□  (2)  Transportable 

□  (3)  Other  (please  specify: _ ) 


Solutions:  Consult  the  proposed  solutions  based  upon  the  item  numbering: 

(1,  2)  Smaller  devices  tend  to  be  lighter  and  easier  to  carry.  However,  size  should  be 
balanced  against  the  display  requirements  (see  device  capabilities). 

(3)  Identify  the  issue  and  consult  information  throughout  this  guide  that  may  assist  in 
overcoming  any  potential  student  challenge  encountered. 
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Description:  This  subfactor  focuses  on  the  ability  and  resources  required  to  support  various  handheld 
mobile  devices  over  time,  including  the  ability  to  maintain  and  upgrade  devices. 


Issues:  Check  the  box  next  to  each  of  the  following  potential  device  supportability  issues  that 

are  identified  below,  where  the  issue  (or  lack  thereof)  may  present  a  challenge  to 
implementing  handheld  mobile  technologies  and  digital  applications  into  the  AIT 
course  under  consideration. 

□  (1)  Access  to  Army's  network  infrastructure 

□  (2)  Service  and  repair  capabilities 

□  (3)  Need  for  upgrades 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  When  access  to  the  Army  network  is  restricted,  an  alternative  solution  might  be 
to  set  up  a  private  network  for  the  device.  This,  of  course,  is  dependent  on 
available  resources  and  capabilities. 

(2)  When  service  and  repair  capabilities  are  limited  or  not  available  for  the  device, 
service  warranties  might  be  an  alternative  option. 

(3)  Be  aware  of  the  lifecycle  of  the  device  and  ensure  device  upgrades  are  compatible 
with  older  applications. 

(1-3)  Identify  the  issue  and  consult  information  throughout  this  guide  that  may  assist 
in  overcoming  any  potential  student  challenge  encountered. 
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Description:  This  subfactor  focuses  on  the  need  to  have  additional  peripherals  (e.g.,  projector)  to  be 
used  in  conjunction  with  the  devices  to  deliver  training. 


Issues:  Check  the  box  next  to  each  of  the  following  potential  device  peripheral  issues  that  are 

identified  below;  where  the  issue  (or  lack  thereof)  may  present  a  challenge  to 
implementing  handheld  mobile  technologies  and  digital  applications  into  the  AIT 
course  under  consideration. 

□  (1)  CAC-access  is  required 

□  (2)  Additional  power  requirements  for  peripherals 

□  (3)  Other  (please  specify: _ ) 


Solutions:  For  those  issues  identified  above,  consult  the  proposed  solutions  based  upon  the  item 

numbering: 

(1)  CAC-enabled  devices  eliminate  the  need  for  encryption  and  provide  users  with 
access  to  the  Army's  network. 

(2)  If  access  to  an  electrical  outlet  is  limited,  it  is  important  to  consider  the  battery 
life  of  the  device  being  selected  and  what  workarounds  may  be  required  (e.g., 
spare  battery;  guidelines  for  conserving  power). 

(3)  Identify  the  issue  and  consult  information  throughout  this  guide  that  may  assist  in 
overcoming  any  potential  student  challenge  encountered. 
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Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints 

- ^  - --7— -  n".  \r  S  V-  yr-yvy  r~,."  ~  f_-  —  -  . 

Factor:  Networking  and  Security 


Subfactor:  Networking  Capabilities,  Restrictions,  and  Policy  Requirements 


Description:  This  subfactor  focuses  on  the  networking  capabilities,  restrictions,  and  policy 
requirements  associated  with  connecting  and  using  handheld  mobile  devices  on  a  network. 


Issues:  The  following  items  are  presented  to  determine  the  extent  to  which  the  mobile 

training  in  question  necessitates  the  following  networking  capabilities.  If  you  check 
any  of  these  items,  then  you  should  proceed  to  the  following  question.  If  you  do  not 
check  any  items,  it  is  likely  that  networking  capabilities,  restrictions,  and  policy 
requirements  are  not  an  issue  for  the  training  in  question. 

Which  of  the  following  network-requiring  capabilities  will  be  required  for  the  handheld 
mobile  technology  or  digital  application  being  considered? 

□  Email 

□  Interactive  play 

□  Real-time  Chat  (non-text  message) 

□  Voice  over  Internet  Protocol  (VoIP) 

□  Video  chat 

□  File  Sharing 

□  Document  Access 

□  Global  Positioning  System  (GPS) 

□  Internet  Access 

□  Other  options  necessitating  3G,  4G,  or  Wi-Fi 

If  you  selected  any  of  these  capabilities,  it  seems  as  if  the  training  in  question 
necessitates  network  access.  The  following  options  are  related  to  networking 
capabilities,  restrictions,  and  policy  requirements  for  the  area  (base)  in  which  training 
is  to  be  conducted.  Policies  and  requirements  may  be  different  from  installation  to 
installation.  Thus,  these  issues  may  or  may  not  exist  universally. 

Which  of  the  following  options  best  describes  the  networking  capabilities,  policies, 
and  restrictions  for  your  implementation  location?  Please  select  only  one  of  the 
following: 

□  (1)  Handheld  mobile  devices  are  permitted  to  access  the  .mil  server. 

□  (2)  Handheld  mobile  devices  are  not  permitted  to  access  the  .mil  server  but  a 

private  network  (WiFi  and/or  mobile  network)  is  currently  available  for 
handheld  mobile  devices. 

□  (3)  Handheld  mobile  devices  are  not  permitted  to  access  the  .mil  server.  A 

private  network  (WiFi  and/or  mobile  network)  is  not  currently  available,  but 
we  are  permitted  to  build  one. 

□  (4)  Handheld  mobile  devices  are  not  permitted  to  access  the  .mil  server.  A 

private  network  (WiFi  and/or  mobile  network)  is  not  currently  available.  It  is 
unknown  if  one  can  be  constructed. 

□  (5)  Handheld  mobile  devices  are  not  permitted  to  access  the  .mil  server. 

Constructing  and  using  a  private  network  (WiFi  and/or  mobile  network)  is  not 
_ permitted. _ 
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Solutions:  For  the  statement(s)  you  have  identified  above,  consult  the  proposed  solutions  based 

upon  the  item  numbering: 

(1)  If  devices  are  given  full  access  to  the  .mil  server,  training  with  handheld  mobile 
devices  will  likely  share  all  the  access  functionality  and  restrictions  currently 
afforded  to  this  network. 

(2)  If  devices  are  given  full  access  to  a  previously  established  secure  or  restricted 
network  (i.e.,  a  non-.mil  network),  training  with  handheld  mobile  devices  may 
share  the  access  functionality  and  restrictions  currently  afforded  to  this  network. 

(3)  If  permission  has  been  granted  to  build  a  secure  or  restricted  network,  special 
attention  needs  to  be  paid  to  the  restrictions  and  policies  set  by  NEC  for  this 
network  build.  Further,  special  consideration  should  be  given  to  potential  time 
investments  related  to  network  construction. 

(4)  If  network  access  is  needed  but  not  permitted,  training  objectives  and  method 
may  have  to  be  changed  if  a  mobile  platform  is  used.  Alternatively,  it  may  be 
the  case  that  a  handheld  mobile  device  is  not  the  best  tool  for  delivering  the 
training  in  question. 

(5)  Based  on  your  previous  responses,  network  access  is  needed  for  your  training. 

At  this  point,  it  is  important  to  determine  if  network  access  is  truly  needed.  If 
network  access  is  needed,  you  should  be  aware  that  attempting  to  negotiate  with 
NEC  to  construct  a  private  network  is  not  always  successful  and  has  been  known 
to  take  as  long  as  6  months.  Specific  policies  and  regulations  can  vary  from 
location  to  location.  Alternatively,  it  may  be  the  case  that  a  handheld  mobile 
device  is  not  the  best  tool  for  delivering  the  training  in  question. 
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Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints 
Factor:  Networking  and  Security 


Subfactor:  Device  and  Application  Security 
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Factor:  Networking  and  Security 
Subfactor:  Student  restrictions 


Description:  This  subfactor  focuses  on  the  restrictions  or  limitations  placed  on  AIT  students  as  part  of 
their  training  program. 


Issues:  Will  students  have  access  to  handheld  mobile  devices  outside  of  the  training 

environment? 

□  Yes  □  No 


Solutions:  If  you  answered  'Yes'  to  the  previous  question,  the  propensity  for  student  misuse  of 

the  device  is  high.  The  following  list  of  solutions  may  help  to  deter  student  misuse. 

If  you  answered  'No',  these  solutions  may  still  be  of  relevance  if  student  use  will  not 

be  actively  monitored  during  training. 

The  following  solutions  may  provide  safe-guards  against  the  student  misuse: 

•  Disable  irrelevant  features  prior  to  delivery. 

•  Password-protect  applications  irrelevant  to  training  using  third-party  application 
softwa  re. 

•  Discuss  acceptable  and  unacceptable  use  with  students  prior  to  device  delivery. 

•  Remind  Soldiers  of  AIT  communication  policies  and  penalties  (such  as  loss  of 
privileges)  related  to  misuse. 

•  Keep  a  record  of  phones  assigned  to  track  individual  misuse  behavior  (such  as 
using  the  phone  feature  on  a  handheld  mobile  device). 

•  Inform  students  of  misuse  safeguards. 


Considerations:  Although  these  security  options  are  available,  the  following  limitations  and 
considerations  may  be  relevant: 

(1)  Permanency  of  restrictions:  Some  security  safeguards  and  disabled  features  are 
easily  circumvented  via  hard  restart  (i.e.,  battery  or  SIM  card  removal)  or  Soldier 
access  to  system  and  software  settings. 

(2)  If  several  device  features  normally  afforded  to  users  are  disabled  and  training 
does  not  necessitate  use  of  the  device  outside  of  a  training  environment,  consider 
storing  devices  in  a  secure  location  instead  of  allowing  students  to  maintain  them 
in  their  possession. 
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Metafactor:  Hardware  and  Infrastructure  Capabilities  and  Constraints 
Factor:  Networking  and  Security 


Subfactor:  Device  Management 
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Metafactor:  Overarching  Factors 


The  following  overarching  factors  concern  estimating  the  training  value  and  costs  associated  with 
integrating  handheld  mobile  technologies  and  digital  applications  into  AIT.  Estimating  value  and  cost 
can  be  extremely  complex  endeavors  and  require  knowledge  of  training  effectiveness  evaluation  (e.g, 
Kirkpatrick,  1996)  and  return  on  investment  calculations  (e.g,  Phillips,  2003). 

This  section  is  intended  to  provide  preliminary  guidance  on  information  that  is  pertinent  to  estimating 
training  value  and  costs  only.  Consult  the  above  references  (Kirkpatrick,  1996;  Philips,  2003)  for 
additional  details  on  developing  ROI  estimates  relevant  for  training  interventions. 

When  estimating  costs  and  values,  use  U.S.  dollar  approximations,  if  possible. 


Factor:  Training  Value 


Training  value  can  be  estimated  through  a  number  of  means  including  student  throughput,  student 
reactions  to  training,  student  knowledge  gains,  student  skill  improvement  and  outcomes  associated 
with  training  goals,  including  performance  on  the  job.  These  approaches  and  their  corresponding 
measures  are  typically  done  after  a  training  program  or  improvement  has  been  implemented.  The 
following  means  of  estimating  value  are  intended  to  provide  insight  into  the  potential  value  of 
handheld  mobile  devices  and  digital  applications  prior  to  their  integration  into  training. 


Subfactor:  Supplement  training  content 


Description:  This  subfactor  focuses  on  the  use  of  handheld  mobile  devices  to  supplement  training  and 
the  program  of  instruction  for  a  course.  As  an  example,  blended  learning  strategies  where  handheld 
mobile  devices  are  integrated  with  formal  classroom  instruction,  hands-on  activities,  or  some  other 
form  of  training  delivery  would  be  considered  as  supplemental  use  of  handheld  mobile  devices. 

Where  the  objective  of  the  use  of  the  handheld  mobile  device  and  application  are  to  supplement 
training  content,  the  following  means  of  estimating  the  value  may  be  applied  when  other  measures 
(see  above)  are  not  readily  available. 

(A)  The  following  metrics  may  provide  some  indication  of  the  value  of  digital  applications  in  AIT.  Fill 
in  the  appropriate  information  below: 

_ Percent  (out  of  100%)  of  total  course  training  content  included  in  application. 

_ Amount  of  time  (in  hours  and  fractions)  student  spend  using  the  application. 

_ Number  of  training  lessons  or  modules  included  in  the  application. 

(B)  Please  indicate  the  principal  use  of  the  application  to  supplement  training: 


(C)  Check  either  of  the  following  that  may  be  applicable  as  further  evidence  of  the  value  of  the 
handheld  mobile  device  and  associated  digital  applications: 

□  The  application  allows  for  additional  practice  on  key  skills  not  available  or  convenient 

through  other  means. 

□  The  application  allows  for  practicing  skills  that  might  otherwise  be  dangerous  to 

practice  with  actual  equipment. 
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You  may  wish  to  write  up  a  short  narrative  summarizing  the  information  presented  here  as 

evidence  of  the  value  of  the  handheld  mobile  device  or  application  to  supplementing  training 
content. 


Subfactor:  Self-development  training  content 


Description:  This  subfactor  focuses  on  the  use  of  handheld  mobile  devices  to  be  used  for  self¬ 
development  training.  This  factor  should  not  be  confused  with  unaided  training,  as  self-development 
training  is  driven  by  student  motivation  to  engage  in  an  activity  beyond  what  is  expected  for  normal 
training  purposes. 

(A)  The  following  metrics  may  provide  some  indication  of  the  value  of  digital  applications  in  AIT.  Fill 
in  the  appropriate  information  below: 

_ Percent  (out  of  100%)  of  total  training  content  included  in  application  for  self¬ 
development. 

_ Amount  of  time  (in  hours  and  fractions)  student  spend  using  the  application  for  self¬ 
development. 

_ Number  of  training  lessons  or  modules  included  in  the  application  for  self¬ 
development. 

(B)  Check  any  of  the  following  that  may  be  applicable  as  further  evidence  of  the  value  of  the 
handheld  mobile  device  and  associated  digital  applications: 

□  The  application  allows  for  additional  practice  on  key  skills  not  available  or  convenient 

through  other  means. 

□  The  application  allows  for  practicing  skills  that  might  otherwise  be  dangerous  to 

practice  with  actual  equipment. 

You  may  wish  to  write  up  a  short  narrative  summarizing  the  information  presented  here  as 
evidence  of  the  value  of  the  handheld  mobile  device  or  application  to  supplementing  training 
content. 


Subfactors:  Demonstration,  assessment/testing 


Description:  This  subfactor  focuses  on  the  development  of  applications  as  a  method  for  demonstrating 
(e.g.,  crew  drills),  assessing  (e.g.,  knowledge  checks),  or  testing  (e.g.,  using  handheld  mobile  devices 
to  test  student  training  content  knowledge  or  skill)  during  learning. 

Demonstration 

(A)  The  following  metrics  may  provide  some  indication  of  the  value  of  digital  applications  in 
AIT.  Fill  in  the  appropriate  information  below: 

_ Percent  (out  of  100%)  of  total  training  objectives  included  in  application  for 

demonstration  purpose. 

_ Amount  of  time  (in  hours  and  fractions)  student  spend  using  the  application  for 

demonstration  purpose. 

You  may  wish  to  write  up  a  short  narrative  summarizing  the  information  presented  here  as 
evidence  of  the  value  of  the  handheld  mobile  device  or  application  to  supplementing  training 
content. 
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Assessment/testing 

The  following  metrics  may  provide  some  indication  of  the  value  of  digital  applications  in  AIT.  Fill  in 
the  appropriate  information  below: 

_ Percent  (out  of  100%)  of  total  training  objectives  that  could  be  assessed  using  the 

application. 

_ Amount  of  time  (in  hours  and  fractions)  student  spend  using  the  application  for 

assessment. 

_ Number  of  training  lessons  or  modules  included  in  the  application  that  could  be 

assessed. 

_ Performance  on  embedded  knowledge  checks  included  in  the  application. 

You  may  wish  to  write  up  a  short  narrative  summarizing  the  information  presented  here  as 
evidence  of  the  value  of  the  handheld  mobile  device  or  application  to  supplementing  training 
content. 


Subfactor:  Reusability 


Description:  This  subfactor  focuses  on  what  aspects  of  mobile  learning  are  reusable.  For  example,  can 
devices  be  used  and  then  returned  to  be  used  by  someone  else.  Also,  can  applications  be  developed 
with  templates  to  reuse  and  save  on  development  costs. 

(A)  The  following  metrics  may  provide  some  indication  of  the  value  of  digital  applications  in  AIT. 
Fill  in  the  appropriate  information  below: 

_ Number  of  courses  for  which  device  could  be  reused  without  significant 

maintenance,  modification,  or  upgrade. 

_ Number  of  courses  for  which  the  application  or  content  from  the  application  could 

be  reused  without  significant  maintenance,  modification,  or  upgrade. 

_ Estimated  cost  savings  (in  dollars)  involved  in  reusing  application  or  content  from 

application  for  other  uses. 

(B) You  may  wish  to  write  up  a  short  narrative  summarizing  the  information  presented  here  as 
evidence  of  the  value  of  the  handheld  mobile  device  or  application  to  supplementing  training 
content. 
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Metafactor:  Overarching  Factor 


Factor:  Training  Cost 


To  assist  in  making  determinations  regarding  the  possible  return  on  investments  involved  in 
integrating  handheld  mobile  technologies  and  digital  applications  into  AIT  you  should  attempt  to  gather 
accurate  information  related  to  the  following  device,  application  and  use  costs,  for  each  of  the 
following.  When  calculating  costs,  gather  information  related  to  initial,  support,  and  maintenance 
costs. 


Subfactor:  Device  Costs 


Description:  This  subfactor  focuses  on  the  costs  associated  with  purchasing  a  device  and  device 
accessories  (including  Device  Peripherals). 

_ (in  $'s)  Estimated  Device  Total  costs  (including  loading  applications  and  reuse,  and 

peripherals) 


Subfactor:  Network  Costs 


Description:  This  subfactor  focuses  on  the  costs  associated  with  using  or  establishing  a  network  for  use 
with  the  handheld  mobile  devices.  This  could  include  the  cost  of  creating  a  local  network  using  WIFI, 
data  plan  costs  that  use  SIM  card  capabilities  to  connect  to  a  network  (ex.  3G). 

_ (in  $'s)  Network  costs 


Subfactor:  Application  Costs 


Description:  This  subfactor  focuses  on  the  costs  associated  with  developing  training  applications  to  be 
used  on  handheld  mobile  devices. 

_ (in  $'s)  Application  development  costs  (including  all  costs  for  implementing  all  uses 

identified  above  under  value) 


Subfactor:  Application  Costs 


Description:  This  subfactor  focuses  on  the  costs  associated  with  maintaining  and  upgrading  devices 
and  applications.  Examples  of  maintenance  and  upgrades  include,  but  are  not  limited  to,  simple 
device  repair  (e.g.,  damaged  screen),  device  operating  system  (OS)  upgrades,  application  bug 
removal,  and  application  upgrades  intended  to  reflect  changes  in  training  content. 

_ (in  $'s)  Device/Application  sustainment  costs  (including  all  costs  for  implementing 

all  uses  identified  above  under  value). 

You  may  wish  to  write  up  a  short  narrative  summarizing  the  cost  information  presented  here  as 
evidence  of  the  cost  of  the  handheld  mobile  device  or  application  to  supplementing  training 
content.  You  may  also  wish  to  provide  an  estimate  of  the  return  on  investment  by  translating 
values  previously  estimated  into  dollar  figures.  An  ROI  index  can  be  estimated  using  the  following 
formula: 

ROI  %  =  Total  Value  ($'s)  -  Total  Costs  ($'s)  x  100% 

Total  Costs  ($'s)  ° 
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For  Additional  Information 


For  additional  information  regarding  the  Connecting  Soldiers  to  Digital  Applications  (CSDA)  project, 
please  consult  the  following  links  (Note:  link(s)  may  not  be  permanent  and  internet  search  may  be 
required): 

http://www.arcic.armv.mil/connectinq-soldiers-to-diqital-applications.html 

For  additional  information  regarding  the  research  effort  in  support  of  the  development  of  this  Practical 
Decision  Guide,  please  see  the  similarly  titled  U.S.  Army  Research  Institute  Technical  Report  in  the 
Defense  Technical  Information  center  (http://www.dtic.mil/dtic/). 

Kirkpatrick,  D.  (1996).  Great  ideas  revisited:  Revisiting  Kirkpatrick's  four  level  model.  Training  and 
Development,  50(1),  54-58. 

Phillips,  J.  J.  (2003).  Return  on  investment  in  training  and  performance  improvement  programs. 
Burlington,  MA:  Butterworth-Heinemann. 
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Glossary 

L _ 


Digital  application 

The  software  designed  and  installed  on  mobile  devices.  Applications  can  be  designed  for  different 
objectives  and  using  different  methods.  Applications  designed  with  the  purpose  of  training  should 
include  training  methods  and  incorporate  training  methods.  < return  to  text> 

Factor 

In  the  guide,  a  factor  refers  to  a  grouping  or  organization  of  considerations  at  a  general  level.  Factors 
are  used  to  provide  information  about  mobile  learning  issues  and  solutions  at  a  general  level.  < return 
to  text> 

Handheld  mobile  technology 

Devices  such  as  smartphones,  tablets,  or  E-readers  that  can  execute  various  software  applications. 

< return  to  text> 

Haptic  interaction 

The  feedback  that  comes  from  a  device  to  a  user  which  usually  consists  of  a  force  (e.g.,  a  vibration) 
against  the  user's  skin.  < return  to  text> 

Mobile  learning 

Involves  the  use  of  mobile,  typically  handheld,  devices  to  present  information  for  the  purpose  of 
learning.  While  learning  takes  place  on  mobile  devices  the  content  is  usually  displayed  via  digital 
applications  installed  on  the  device.  < return  to  text> 

Metafactor 

In  the  guide,  a  metafactor  refers  to  a  grouping  or  organization  of  consideration  at  the  most  general 
level.  The  metafactors  are  best  used  to  organize  information  as  it  requires  a  deeper  understanding 
(i.e.  the  factor  level)  to  begin  to  understand  important  considerations.  < return  to  text> 

Operating  system 

A  collection  of  software  that  manages  the  hardware  on  a  computer  system.  There  are  multiple  mobile 
device  operating  systems,  such  as  Android®  or  iOS®  and  the  design  and  development  of  applications 
for  each  system  is  different.  < return  to  text> 

Podcast/ Vodcast 

Audio  or  video  online  media  typically  containing  conversation  or  simple  communication  to  address 
topical  issues.  < return  to  text> 

Software  Development  Kit  (SDK) 

A  grouping  of  software  development  tools  typically  arranged  into  a  graphic  user  interface  allowing  for 
easy  development  of  applications  or  software.  < return  to  text> 

Level  of  fidelity  of  training 

This  describes  the  relationship  between  training  context  and  the  context  in  the  real-world  when  what 
has  been  trained  will  need  to  be  used.  High  level  of  fidelity  in  training  would  mean  that  training  would 
be  very  similar  to  the  actual  or  real-world  situation  that  learners  would  confront.  < return  to  text> 

Training  developer  and  instructional  designer 
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The  person/people  responsible  for  the  development  of  content  and  content  delivery  systems,  including 
information  technology  experts  who  are  used  to  create  applications  to  deliver  content.  < return  to 
text> 

Subfactor 

In  this  guide,  this  is  the  most  detailed  and  specific  level  of  understanding  and  representation  of  the 
considerations  related  to  developing  and  implementing  mobile  learning.  < return  to  text> 

SIM  Card 

Storage  device  in  mobile  telephones  that  is  used  to  store  data,  applications,  and  other  software  that 
effects  the  operation  of  the  phone.  < return  to  text> 
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